Method and apparatus for transferring remote session data

ABSTRACT

Examples of systems and methods are provided for communication and for forwarding display data related to a remote session between a client device and a remote server to a host device. The system may facilitate establishing the remote session with the remote server. The system may facilitate establishing a trusted relationship between the client device and the host device. The system may filter out data related to local graphical user interface (GUI) and selectively forward from the client device to the host device display data related to the remote session established between the client device and the remote server.

CROSS-REFERENCES TO RELATED APPLICATIONS

The present application claims the benefit of priority under 35 U.S.C.§119 from U.S. Provisional Patent Application Ser. No. 61/169,664,entitled “ENABLING SERVER SIDE COMPUTING ON A REMOTE CLIENT WHILEFACILITATING AN IMPROVED USER EXPERIENCE FOR THE REMOTE CLIENT USER,”filed on Apr. 15, 2009, and U.S. Provisional Application Ser. No.61/169,667, entitled “ENABLING SERVER SIDE COMPUTING FROM A REMOTECLIENT AND FACILITATING THE REMOTE CLIENT TO CUSTOMIZE AND CONTROL ASERVER APPLICATION,” filed on Apr. 15, 2009, both of which are herebyincorporated by reference in their entirety for all purposes.

BACKGROUND

One approach to the design and implementation of computer networks,particularly with regard to the development of client/serverapplications, includes designing client applications and client devicesso that the majority of the heavily used resources are at a remotecomputing device, such as a centralized server, connected via a network.These client devices generally have minimal memory, disk storage, andprocessor power, but are designed under the premise that most usersconnected to a powerful server do not need the additional processingpower. With these client devices, the total cost of ownership isminimized, because of the reduced resources and because the clients canbe centrally administered and updated from the server.

In practice, because a server allows a user logging from a remote clientdevice to access applications on the server, a user is required toestablish his identity through an authentication mechanism. Well-knownauthentication mechanisms may include username/password combination,verification of biometric characteristic of the user (fingerprint,retina scan, etc.), removable security modules such as dongles andsmartcards, radio frequency identification (RFID) tags, and others.

These authentication techniques suffer from drawbacks that they may becumbersome, may be perceived by some users as being intrusive (biometricscans) or may be expensive to implement because they require specialsensors, software and/or hardware. Furthermore, these authenticationsmay limit portability of a remote desktop session from one client deviceto another. A better method for authentication and portability of aremote session is needed.

SUMMARY

In an aspect of the disclosure, a system may be provided at a clientdevice side for communication and for forwarding display data related toa remote session between a client device and a remote server to a hostdevice. The system may comprise one or more of: a client remote sessionmodule configured to facilitate establishing the remote session with theremote server, a client trusted relationship module configured tofacilitate establishing a trusted relationship between the client deviceand the host device; and a client forwarding module configured to filterout data related to local graphical user interface (GUI) and selectivelyforward from the client device to the host device display data relatedto the remote session established between the client device and theremote server.

In another aspect of the disclosure, a method may be provided at aclient device side for communication and for forwarding display datarelated to a remote session between a client device and a remote serverto a host device. The method may comprise one or more of the following:facilitating establishing the remote session with the remote server;facilitating establishing a trusted relationship between the clientdevice and the host device, and filtering out data related to localgraphical user interface (GUI), and selectively forwarding, from theclient device to the host device, display data related to the remotesession established between the client device and the remote server.

In yet another aspect of the disclosure, a machine-readable medium maybe encoded with instructions for execution at a client device side forcommunication and for forwarding display data related to a remotesession between a client device and a remote server to a host device.The instructions may comprise code for one or more of the following:facilitating establishing the remote session with the remote server,facilitating establishing a trusted relationship between the clientdevice and the host device and filtering out data related to localgraphical user interface (GUI), and selectively forwarding, from theclient device to the host device, display data related to the remotesession established between the client device and the remote server.

In yet another aspect of the disclosure, an apparatus may be providedfor communication and for forwarding display data related to a remotesession between a client device and a remote server to a host device.The apparatus may comprise one or more of the following: means forfacilitating establishing the remote session with the remote server,means for facilitating establishing a trusted relationship between theclient device and the host device, and means for filtering out datarelated to local graphical user interface (GUI), and means forselectively forwarding, from the client device to the host device,display data related to the remote session established between theclient device and the remote server.

In yet another aspect of the disclosure, a system may be provided at ahost device side for displaying graphical user interface (GUI) for aremote session between a client device and a remote server. The systemmay comprise one or more of the following: a host trusted relationshipmodule configured to facilitate establishing a trusted relationshipbetween the client device and the host device, a host display datamodule configured to receive, from the client device, display data forthe remote session established between the client device and the remoteserver, the display data based on a remote session protocol format, anda host remote session view module configured to facilitate displaying aremote application view based on the received display data for theremote session, wherein the host device is configured to communicatewith the client device via a wireless connection.

In yet another aspect of the disclosure, a method may be provided at ahost device side for communication and for displaying graphical userinterface (GUI) for a remote session between a client device and aremote server. The method may comprise one or more of the following:facilitating establishing a trusted relationship between the clientdevice and the host device, receiving, from the client device, displaydata for the remote session established between the client device andthe remote server, the display data based on a remote session protocolformat, and facilitating displaying a remote application view based onthe received display data for the remote session, wherein the hostdevice is configured to communicate with the client device via awireless connection.

In yet another aspect of the disclosure, a machine-readable medium maybe encoded with instructions for execution at a host device side forcommunication and for displaying graphical user interface (GUI) for aremote session between a client device and a remote server. Theinstructions may comprise code for one or more of the following:facilitating establishing a trusted relationship between the clientdevice and the host device, receiving, from the client device, displaydata for the remote session established between the client device andthe remote server, the display data based on a remote session protocolformat, and facilitating displaying a remote application view based onthe received display data for the remote session, wherein the hostdevice is configured to communicate with the client device via awireless connection.

In yet another aspect of the disclosure, an apparatus may be providedfor communication and for displaying graphical user interface (GUI) fora remote session between a client device and a remote server. Theapparatus may comprise one or more of the following: means forfacilitating establishing a trusted relationship between the clientdevice and the host device, means for receiving, from the client device,display data for the remote session established between the clientdevice and the remote server, the display data based on a remote sessionprotocol format, and means for facilitating displaying a remoteapplication view based on the received display data for the remotesession, wherein the host device is configured to communicate with theclient device via a wireless connection.

It is understood that other configurations of the subject technologywill become readily apparent to those skilled in the art from thefollowing detailed description, wherein various configurations of thesubject technology are shown and described by way of illustration. Aswill be realized, the subject technology is capable of other anddifferent configurations and its several details are capable ofmodification in various other respects, all without departing from thescope of the subject technology. Accordingly, the drawings and detaileddescription are to be regarded as illustrative in nature and not asrestrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates an example of a display on a client device during aremote access session with a remote server.

FIG. 1B illustrates an example of a web browser running on a server.

FIGS. 1C to 1E illustrate examples of different portions of the webbrowser displayed on a client device.

FIG. 2 is a conceptual block diagram of a computer network according tocertain aspects of the present disclosure.

FIG. 3 is a conceptual block diagram of a server according to certainaspects of the present disclosure.

FIG. 4 is a conceptual block diagram of a client device according tocertain aspects of the present disclosure.

FIG. 5A is a conceptual block diagram of a server according to certainaspects of the present disclosure.

FIG. 5B is a conceptual block diagram of a client according to certainaspects of the present disclosure.

FIG. 6 illustrates an example of a remote access connection.

FIG. 7 illustrates a composite view at the client device according tocertain aspects of the present disclosure.

FIG. 8A illustrates an example of a local GUI according to certainaspects of the present disclosure.

FIG. 8B illustrates an example of a graphical keyboard according tocertain aspects of the present disclosure.

FIG. 9 illustrates a display at the client device according to certainaspects of the present disclosure.

FIG. 10 is a conceptual block diagram of server and client remote accessmodules according to certain aspects of the present disclosure.

FIG. 11 illustrates an example of a plurality of connections between theserver and the client device.

FIG. 12 is a block diagram that illustrates an exemplary computingsystem in accordance with certain embodiments of the present disclosure.

FIG. 13A illustrates an example of an operation of a system according toone aspect of the present disclosure.

FIG. 13B illustrates an example of a configuration of an apparatusaccording to one aspect of the present disclosure.

FIG. 14A illustrates an example of a remote application displayed at aclient device.

FIG. 14B illustrates an example of a graphical keyboard displayed at theclient device for entering text for the remote application.

FIGS. 15A illustrates an example of an area of the display output of aremote application that is displayed in a remote application view of aclient device.

FIG. 15B illustrates an example in which the area of the display outputin FIG. 15A is scrolled to the right.

FIG. 16A illustrates an example in which a dialog box or window islocated outside the area of the display output displayed in the remoteapplication view.

FIG. 16B illustrates an example in which the area of the display outputdisplayed in the remote application view is scrolled to the right toencompass the dialog box or window.

FIG. 17 illustrates an example in which a portion of an object of aremote application is displayed at a client device.

FIG. 18 illustrates an example in which a remote application view of theclient device is scrolled to the right to reveal the entire object ofthe remote application.

FIG. 19A illustrates an example in which a portion of an object islocated within an area of the display output displayed in the remoteapplication view.

FIG. 19B illustrates an example in which the entire object is locatedwithin the area of the display output displayed in the remoteapplication view.

FIG. 20 illustrates an example of a local GUI at a client device listingapplications that are available at a remote server.

FIG. 21 illustrates an example of a remote desktop displayed at a clientdevice.

FIG. 22 is a conceptual block diagram of an agent module according tocertain aspects of the present disclosure.

FIG. 23A illustrates an example of an operation of an apparatusaccording to one aspect of the present disclosure.

FIG. 23B illustrates an example of a configuration of an apparatusaccording to one aspect of the present disclosure.

FIG. 24A illustrates an example of an operation of an apparatusaccording to one aspect of the present disclosure.

FIG. 24B illustrates an example of a configuration of an apparatusaccording to one aspect of the present disclosure.

FIG. 25 is a block diagram of a communication network in accordance withcertain configurations of the present disclosure.

FIG. 26A is a conceptual block diagram that illustrates exemplarymodules implemented at a client device in accordance with certainconfigurations of the present disclosure.

FIG. 26B is a conceptual block diagram that illustrates exemplarymodules implemented at a client device in accordance with certainconfigurations of the present disclosure.

FIG. 26C is a conceptual block diagram that illustrates exemplarymodules implemented at a client device in accordance with certainconfigurations of the present disclosure.

FIG. 26D is a conceptual block diagram that illustrates exemplarymodules implemented at a client device in accordance with certainconfigurations of the present disclosure.

FIG. 26E is a conceptual block diagram that illustrates exemplarymodules implemented at a client device in accordance with certainconfigurations of the present disclosure.

FIG. 27A is a conceptual block diagram that illustrates exemplarymodules implemented at a host device in accordance with certainconfigurations of the present disclosure.

FIG. 27B is a conceptual block diagram that illustrates exemplarymodules implemented at a host device in accordance with certainconfigurations of the present disclosure.

FIG. 27C is a conceptual block diagram that illustrates exemplarymodules implemented at a host device in accordance with certainconfigurations of the present disclosure.

FIG. 28 is a flow chart illustrating an example of an operation ofauthenticating a host device for establishment of a remote session.

FIG. 29 is a flow chart illustrating an example of an operation ofacquiring authentication from a client device and starting a remotesession with a remote server.

FIG. 30 is a flow chart illustrating an example of an operation ofsustaining a remote session with a remote server.

FIG. 31 is a flow chart illustrating an example of an operation ofhand-off of a remote session to a host device.

FIG. 32 is a flow chart illustrating an example of an operation ofaccepting a hand-off of a remote session from a client device.

FIG. 33 is a flow chart illustrating an example of an operation offorwarding display data for a remote session to a host device.

FIG. 34 is a flow chart illustrating an example of an operation ofaccepting forwarded display data for a remote session from a clientdevice.

FIG. 35 is a flow chart illustrating an example of an operation ofaccepting a hand-off of a remote session from a host device.

FIG. 36 is a flow chart illustrating an example of an operation ofhand-off of a remote session to a client device.

FIG. 37 is a block diagram of a communication network in accordance withcertain embodiments of the present disclosure.

FIG. 38 is a message exchange diagram illustrating exemplary messagesexchanged in authenticating a remote session.

FIG. 39 is a message exchange diagram illustrating exemplary messagesexchanged in operation of hand-off of a remote session to a host device.

FIG. 40 is a message exchange diagram illustrating exemplary messagesexchanged in displaying a remote session on a host device.

FIG. 41 is a message exchange diagram illustrating exemplary messagesexchanged in operation of hand-off of a remote session to a clientdevice.

FIG. 42 illustrates an example of an operation of an apparatus accordingto one aspect of the present disclosure.

FIG. 43 illustrates an example of an operation of an apparatus accordingto one aspect of the present disclosure.

FIG. 44 illustrates an example of a configuration of an apparatusaccording to one aspect of the present disclosure.

FIG. 45 illustrates an example of a configuration of an apparatusaccording to one aspect of the present disclosure.

FIG. 46 illustrates an example of an operation of an apparatus accordingto one aspect of the present disclosure.

FIG. 47 illustrates an example of a configuration of an apparatusaccording to one aspect of the present disclosure.

FIG. 48 illustrates an example of an operation of an apparatus accordingto one aspect of the present disclosure.

FIG. 49 illustrates an example of an operation of an apparatus accordingto one aspect of the present disclosure.

FIG. 50 illustrates an example of a configuration of an apparatusaccording to one aspect of the present disclosure.

FIG. 51 illustrates an example of a configuration of an apparatusaccording to one aspect of the present disclosure.

FIG. 52 illustrates an example of an operation of an apparatus accordingto one aspect of the present disclosure.

FIG. 53 illustrates an example of an operation of an apparatus accordingto one aspect of the present disclosure.

FIG. 54 illustrates an example of a configuration of an apparatusaccording to one aspect of the present disclosure.

FIG. 55 illustrates an example of a configuration of an apparatusaccording to one aspect of the present disclosure.

DETAILED DESCRIPTION

The detailed description set forth below is intended as a description ofvarious configurations of the subject technology and is not intended torepresent the only configurations in which the subject technology may bepracticed. The appended drawings are incorporated herein and constitutea part of the detailed description. The detailed description includesspecific details for the purpose of providing a thorough understandingof the subject technology. However, it will be apparent to those skilledin the art that the subject technology may be practiced without thesespecific details. In some instances, well-known structures andcomponents are shown in block diagram form in order to avoid obscuringthe concepts of the subject technology. Like components are labeled withidentical element numbers for ease of understanding.

FIG. 1A shows an example of a display 100 at a client device with alarge screen during a remote access session with a server. The display100 includes the local desktop 105 of the client device, a remote viewwindow 115 showing an image of a remote desktop 115 and a remote webbrowser application 120 running on the server. The image of the remotedesktop 115 and remote application 120 are based on display output dataof the remote desktop 115 and remote application 120 from the server.

FIG. 1B shows an enlarged view of the remote web browser application120. The remote web browser application 120 includes a GUI 122 (e.g.,toolbars) and a window 124 for displaying web content. The GUI 122 isdesigned to be viewed on a large display. As a result, it can be verydifferent for a user at a client device with a small display to remotelyinteract with the remote web browser 120. For example, if the image ofthe entire remote web browser application 120 is displayed on a clientdevice with a small display, then the GUI 122 is greatly reduced in sizemaking it difficult for the user to view the GUI 122 and selectgraphical control objects in the GUI 122.

If the user at the client device zooms in on a portion of the remote webbrowser application 120 to enlarge that portion for viewing on a clientdevice with a small screen (e.g., mobile phone), then the user needs toconstantly manipulate the remote view by adjusting the zoom andscrolling to view different portions of the web browser application 120.FIG. 1C shows an example of a client device 135 in which a portion ofthe remote web browser application 120 has been zoomed in for viewing onthe display 140 of the client device 135 (e.g., smartphone). In thisexample, only a portion of the GUI 122 is visible. As a result, the userat the client device has to adjust the zoom and/or scroll the remoteview on the display 140 to view the GUI 122. FIG. 1D shows anotherexample in which a different portion of the GUI 122 is visible on thedisplay 140. FIG. 1E shows an example in which the GUI 122 is notvisible at all on the display 140. The different portions of the webbrowser application 120 displayed in FIGS. 1C to 1E are shown in FIG. 1Bin dashed boxes. Thus, interacting with a remote application running ona server from a client device (e.g., smartphone) can be very cumbersomeand difficult.

FIG. 2 illustrates a simplified diagram of a system 200 in accordancewith an aspect of the present disclosure. The system 200 may include oneore more remote client devices 202 in communication with a servercomputing device 204 (server) via a network 206. In one aspect, theserver 204 is configured to allow remote sessions (e.g., remote desktopsessions) wherein users can access applications and files on the server204 by logging onto the server 204 from a client device 202. Such aconnection may be established using any of several well-known techniquessuch as the Remote Desktop Protocol (RDP) on a Windows-based server.

By way of illustration and not limitation, in one aspect of thedisclosure, stated from a perspective of a server side (treating aserver as a local device and treating a client device as a remotedevice), a server application is executed (or runs) at a server 204.While a remote client device 202 may receive and display a view of theserver application on a display local to the remote client device 202,the remote client device 202 does not execute (or run) the serverapplication at the remote client device 202. Stated in another way froma perspective of the client side (treating a server as remote device andtreating a client device as a local device), a remote application isexecuted (or runs) at a remote server 204.

By way of illustration and not limitation, a client device 202 canrepresent a computer, a mobile phone, a laptop computer, a thin clientdevice, a personal digital assistant (PDA), a portable computing device,or a suitable device with a processor. In one example, a client device202 is a smartphone (e.g., iPhone, Android phone, Blackberry, etc.). Incertain configurations, a client device 202 can represent an audioplayer, a game console, a camera, a camcorder, an audio device, a videodevice, a multimedia device, or a device capable of supporting aconnection to a remote server. In one example, a client device 202 canbe mobile. In another example, a client device 202 can be stationary.According to one aspect of the disclosure, a client device 202 may be adevice having at least a processor and memory, where the total amount ofmemory of the client device 202 could be less than the total amount ofmemory in a server 204. In one example, a client device 202 does nothave a hard disk. In one aspect, a client device 202 has a displaysmaller than a display supported by a server 204. In one aspect, aclient device may include one or more client devices.

In one aspect, a server 204 may represent a computer, a laptop computer,a computing device, a virtual machine (e.g., VMware® Virtual Machine), adesktop session (e.g., Microsoft Terminal Server), a publishedapplication (e.g., Microsoft Terminal Server) or a suitable device witha processor. In one aspect, a server 204 can be stationary. In anotheraspect, a server 204 can be mobile. In certain configurations, a server204 may be any device that can represent a client device. In one aspect,a server 204 may include one or more servers.

In one example, a first device is remote to a second device when thefirst device is not directly connected to the second device. In oneexample, a first remote device may be connected to a second device overa communication network such as a Local Area Network (LAN), a Wide AreaNetwork (WAN), and/or other network.

When a client device 202 and a server 204 are remote with respect toeach other, a client device 202 may connect to a server 204 over anetwork 206, for example, via a modem connection, a LAN connectionincluding the Ethernet or a broadband WAN connection including DSL,Cable, T1, T3, Fiber Optics, Wi-Fi, or a mobile network connectionincluding GSM, GPRS, 3G, WiMax or other network connection. A network206 can be a LAN network, a WAN network, a wireless network, theInternet, an intranet or other network. A network 206 may include one ormore routers for routing data between client devices and/or servers. Aremote device (e.g., client device, server) on a network may beaddressed by a corresponding network address, such as, but not limitedto, an Internet protocol (IP) address, an Internet name, a WindowsInternet name service (WINS) name, a domain name or other system name.These illustrate some examples as to how one device may be remote toanother device. But the subject technology is not limited to theseexamples.

According to certain aspects of the present disclosure, the terms“server” and “remote server” are generally used synonymously in relationto a client device, and the word “remote” may indicate that a server isin communication with other device(s), for example, over a networkconnection(s).

According to certain aspects of the present disclosure, the terms“client device” and “remote client device” are generally usedsynonymously in relation to a server, and the word “remote” may indicatethat a client device is in communication with a server(s), for example,over a network connection(s).

In one aspect of the disclosure, a “client device” may be sometimesreferred to as a client or vice versa. Similarly, a “server” may besometimes referred to as a server device or vice versa.

In one aspect, the terms “local” and “remote” are relative terms, and aclient device may be referred to as a local client device or a remoteclient device, depending on whether a client device is described from aclient side or from a server side, respectively. Similarly, a server maybe referred to as a local server or a remote server, depending onwhether a server is described from a server side or from a client side,respectively. Furthermore, an application running on a server may bereferred to as a local application, if described from a server side, andmay be referred to as a remote application, if described from a clientside.

In one aspect, devices placed on a client side (e.g., devices connecteddirectly to a client device(s) or to one another using wires orwirelessly) may be referred to as local devices with respect to a clientdevice and remote devices with respect to a server. Similarly, devicesplaced on a server side (e.g., devices connected directly to a server(s)or to one another using wires or wirelessly) may be referred to as localdevices with respect to a server and remote devices with respect to aclient device.

FIG. 3 illustrates a simplified block diagram of a server 204 inaccordance with an aspect of the present disclosure. The server 204comprises an agent module 322, an OS module 324, one or more applicationcontrol modules 325, a desktop module 328, a server remote access module334 and an operating system (OS) 330 in communication with the modules.In one aspect, the OS module 324 can be a part of the OS 330. The server204 is communicatively coupled with the network 206 via a networkinterface 332. The modules can be implemented in software, hardwareand/or a combination of both. Features and functions of these modulesaccording to various aspects are further described in the presentdisclosure.

FIG. 4 illustrates a simplified block diagram of a client device 202 inaccordance with an aspect of the present invention. The client device202 comprises a viewer controller (VC) module 456, a client remoteaccess module 452 and an operating system (OS) 460 in communication withthe modules. The modules are further in communication with various userinterface devices (not shown in FIG. 4) via a human interface devices(HID) connection 458. The user interface devices may include one or moreoutput devices (e.g., one or more of a display, a speaker, or otheraudio, image or video output devices) and one or more input devices(e.g., one or more of a keyboard, a mouse, a trackball, a microphone, astylus, a touch screen, a touch pad, a pen, a tablet, or other audio,image or video input devices). The modules are also in communicationwith the network 206 via a network connection 462. The modules can beimplemented in software, hardware and/or a combination of both.Additional features and functions of these modules according to variousaspects of the present disclosure are further described in thedisclosure.

FIGS. 5A and 5B are conceptual block diagrams illustrating a server 204and a client device 202 according to an aspect of the disclosure.

Referring to FIG. 5A, the server 204 may comprise the server remoteaccess module 334 and the agent module 322. The server remote accessmodule 334 is configured to establish a remote access connection 525with the client device 202 to provide the client device 202 with remoteaccess to a desktop and/or applications running on the server 204. Theserver remote access module 334 may comprise a remote access applicationand may communicate with the client device 202 over a network 206 basedon a remote access protocol (e.g., RDP/ICA), other protocols or acombination of protocols. Remote access applications allow a user at aclient device to remotely access a desktop and/or application running ona server. Examples of remote access applications include, but are notlimited to, the Microsoft® Remote Desktop Protocol (RDP) application andthe Citrix® Independent Computing Architecture (ICA) application.

The server remote access module 334 is also configured to receivecommand messages from the client device 202 and communicate the receivedcommand messages to the agent module 322. The agent module 322 may beconfigured to control a desktop and/or application running on the server204 based on the received command messages, as described further below.

The server 204 further comprises an application module 326, the desktopmodule 328 and the OS module 324. The application module 326 comprisesan application 327 and an application control module 325. An application327 may include one or more applications. The application 327 mayinclude, for example, a web browser application, a word processingapplication, a spreadsheet application, a game application, an audioapplication, and/or other applications. The application 327 may alsoinclude applications for rendering multi-media content including, forexample, Adobe® Acrobat, Microsoft® Silverlight, and/or otherapplications. In one example, the application 327 is executed (or runs)at the server 204, and the client device 202 does not execute (or run)the application 327 locally at the client device 202.

An application control module 325 may include one or more applicationcontrol modules. The application control module 325 may be configured tocontrol the application 327 based on commands from the agent module 322.The application control module 325 may comprise one or more ActiveXcontrollers for controlling Windows-based applications or other types ofcontroller. The application control module 325 may also control theapplications based on user inputs received by the remote access module334 from the client device 202. The user inputs may comprise pointerinputs, keyboard inputs and/or other inputs. The application controlmodule 325 may also control the applications based on events generatedby the application 327 or the OS 330.

The desktop module 328 may be configured to generate a desktop thatprovides a GUI for allowing a user to select and launch applications onthe server 204, manage files, and configure certain features of theoperating system 330. The GUI may be a server-based GUI presented by theagent module 322. Alternatively, the GUI may be controlled by the agentmodule 322 and displayed natively on the client device. The desktop maydisplay icons representing different applications that can be launchedon the server 204, in which a user can select an application by moving apointer to the corresponding icon and double clicking on the icon. TheOS module 324 may be configured to communicate certain OS events to theagent module 322 and the server remote access module 334, as describedfurther below.

The server 204 further comprises the operating system 330, which maymanage and provide communication to and receive communication from themodules and application shown in FIG. 5A. The operating system 330 mayalso manage multiple remote user sessions allowing multiple clientdevices to remotely access the server 204. The operating system 330 mayinclude Microsoft® Windows, Linux, Unix, Mac OS or another operatingsystem. Although the server remote access module 334 is shown separatelyfrom the operating system 330, the server remote access module 334 maybe considered part of the operating system 330. For example, the remoteaccess module 334 may comprise an RDP application that comespre-installed on the server 204 as part of Microsoft® Windows.

Referring now to FIG. 5B, the client device 202 may comprise the clientremote access module 452 and the viewer controller module 456. Theclient remote access module 452 may be configured to communicate withthe server 204 over a network 206 to remotely access a desktop and/orapplications running on the server 204. The client remote access module452 may communicate with the server remote access module 334 based on aremote access protocol (e.g., RDP/ICA), other protocols or a combinationof protocols. In one aspect, the client remote access module 452 may beconfigured to receive display output data of a desktop and/orapplication running on the server 204 from the server remote accessmodule 334 over a network 206.

In one aspect of the disclosure, display output data may comprisevisual/audio information that a user located at a server 204 would haveseen on a display at the server (e.g., a “virtual” display) and/or wouldhave heard from an audio output such as a speaker at the server (e.g., a“virtual” speaker). In one aspect, the display output data may comprisean image (e.g., bitmap) and/or drawing commands of the display output ofa desktop and/or application running on the server 204. In one aspect,drawing commands provide instructions or information to a display systemas to what is to be drawn on a display. The instructions or informationmay include, for example and without limitation, the location, texture,geometry, shading, light intensity, or color, or any combinationtherein, of a pixel or group of pixels of the display. In some aspects,a drawing command corresponds to multiple pixels. Still in some aspects,a drawing command corresponds to a single pixel of a display. In someaspects, drawing commands may include raster operations. In one aspectof the disclosure, the phrase “display output data” may be sometimesreferred to as “display data”, “display output” or vice versa.

The client remote access module 452 may be configured to send userinputs to the server remote access module 334 over a network 206. Theuser inputs may comprise pointer inputs, keyboard inputs and/or othertypes of input. The client remote access module 452 may also beconfigured to send command messages to the server remote access module334, which are described in further detail below. The remote accessconnection 525 between the server 204 and client device 202 may comprisea plurality of virtual channels for communicating different types ofdata. For example, a RDP connection may include different virtualchannels for display output data and user inputs. FIG. 6 illustrates anexample of the remote access connection 525 comprising multiple virtualchannels 620 a to 620 c.

The viewer controller module 456 may comprise a remote view module 535,a local view module 540 and a display module 555. The remote view module535 may be configured to receive display output data of a desktop and/orapplication 327 running on the server 204 from the client remote accessmodule 452 and generate a remote view of the display output data of theremote desktop and/or application. In one aspect of the disclosure, thephrase “remote view” may sometimes be referred to as “remote applicationview” or vice versa. In one aspect of the disclosure, the term “anapplication” or “a remote application” may refer to an application, aremote application, a desktop, or a remote desktop.

The local view module 540 may be configured to generate a local GUI thatallows a user at the client device 202 to remotely control theapplication 327 running on the server 204. The local view module 540 maygenerate the local GUI based on a configuration file 541. In one aspect,the file 541 may be generated by the agent module 322 or applicationmodule 326 and be sent to the client device 202. The file 541 may fullydefine the layout, design and logical operation of the local GUI. Thiswould allow the server 204 to update the local GUI in the client device202 without having to make modifications to the client device 202. Thelocal GUI may provide similar controls as the GUI of the application 327running on the server 204. The local GUI may include graphical controlobjects for controlling certain functions of the application 327.

For the example of a web browser application, the graphical controlobjects of the local GUI may include an address field for entering theaddress (e.g., URL) of a web page, a refresh button, a go button, andscrollbars. The local GUI is displayed on the local display 560 of theclient device 202. A user at the client device 202 may select graphicalcontrol objects of the local GUI displayed on the display 560 using aninput device 565, for example, a touch screen overlying the display 560.As described further below, the local GUI may be optimized for thedisplay 560 of the client device 202 to provide the user at the clientdevice 202 with a user friendly interface for remotely controlling theapplication 327 running on the server 204.

The display module 555 may be configured to receive a local GUI from thelocal view module 540 and a remote application view of a remote desktopand/or a remote application from the remote view module 535 and generatea composite view comprising the local GUI and the remote applicationview of the remote desktop and/or remote application.

FIG. 7 shows an example of a composite view rendered on a display 560 ofa client device 202. In this example, the composite view includes aremote application view 704 for viewing an image of the display outputof a remote application 327 running on the server 204. The remoteapplication view is based on the display output data of the remoteapplication 327 received from the server 204. The user may manipulatethe image of the remote application 327 shown in the remote applicationview 704. For example, the remote application view 704 may be responsiveto, for example, viewing selections by a user such as zooming factor,the depth of the color palette used, viewing angle (pan and tilt) and soon. The viewing selections may be provided in a GUI. The user may alsozoom in and out within the remote application view 704 using anytechnique. For example, the user may zoom in by sliding two fingers onthe display 560 away from each other and zoom out by sliding the fingerson the display towards each other on the display 560, which is currentlysupported by, for example, iPhone and Palm Pre. User inputs within theremote application view 704 are directed to the remote view module 535.

The composite view may also include a local GUI 702 and 706. The localGUI 702 may display a local header for the application shown in thedisplay area 704. In one aspect, the local header may display anapplication name based on information received from the server 204. Inanother aspect, the local header may be locally generated (e.g., aremote session window). In yet another aspect, the local header maydisplay an editable text input area, such as an address field for a webbrowser. The local GUI 706 may display, for example, local control GUIobjects such as a local control GUI toolbar. The local control GUIobjects may be based on information received from the remote server 204.

The dimensions and layouts of the remote application view 704 and localGUI 702 and 706 are not limited to the example shown in FIG. 7. Theremote application view 704 and local GUI 702 and 706 may have otherdimensions and layouts, for example, based on the remote applicationbeing accessed by the client device 202.

In one aspect, the local GUI 702 and 706 is not generated from thedisplay output of a remote application 327 running on the server 204.For example, the local GUI 702 and 706 is not an image of the remoteapplication's GUI generated from the display output of the remoteapplication 327 running on the server 204. Instead, the local GUI islocally generated by the local view module 540, for example, based onconfiguration file 541 (shown in FIG. 5B). Furthermore, theconfiguration file 541 may be received by the local view module 540 bythe application control module 325 or the agent module 322.

FIG. 8A shows an example of a local control GUI 706. The local controlGUI 706 comprises a toolbar 806 including graphical control objects. Thegraphical control objects may include a keyboard icon 820, a BACK arrow822, a FORWARD arrow 824, a pointer icon 826, a MOUSE icon 828 and aTOOLS icon 830. When a user selects the keyboard icon 820 (e.g., bytouching or tapping), the local view module 540 may change the localcontrol GUI 806 to display a graphical keyboard 820 a for the user toenter text (shown in FIG. 8B). The layout details of a keyboard 820 amay be provided in a configuration file 541 (shown in FIG. 5B), or maybe based on the local GUI tools provided by the OS 460 of the clientdevice 202.

When a user selects the BACK icon 822, the web browser applicationdisplays a previously displayed web page. In certain embodiments, theclient device 202 may perform this by communicating to the server 204, acontrol code corresponding to this event. In other embodiments, theclient device 202 may include a local cache of previously displayed webpages and may use the cached data to render the previously displayed webpage. In a manner similar to the BACK icon 822, the FORWARD icon 824 maybe used to render a later viewed web page in the web browser. Thepointer icon 826 may allow a user to activate a pointer within thedisplay area of the remote application view 704. The MOUSE icon 828 mayallow a user to activate a mouse cursor, which then will invoke mousemovements or mouse clicks, and the TOOLS icon 830 may open further menuscontaining additional tools, as provided in a configuration file and asprovided by the OS 460 of the client device 202.

It will be appreciated by one skilled in the art that, according to oneaspect of the disclosure, because the control objects of the local GUIare rendered locally, these objects can be advantageously rendered atthe resolution of the local display 560 and can be rendered to havedimensions that can be easily navigated or handled by a user using afinger, a stylus or any other native navigation method(s) provided bythe client device 202 (e.g., track-ball, keyboard, etc.). Furthermore,the available display area for the remote application view (e.g., 704)of a remote application can be fully dedicated to content displayed bythe remote application by deactivating control GUI objects from theremote application view 704. For example, a web browser executed at aremote server 204 may be displayed in the remote application view 704after deactivating scrollbars and menu header of the web browser at theserver 204. The toolbar and the menu header controls can be provided bythe local GUI instead.

FIG. 9 illustrates an example of a composite view 900 that may berendered locally on display 560. The composite view 900 comprises thelocal GUI 902 and 906 and the remote application view 904 of a remoteapplication 327 running on the server 204. In this example, thegraphical control objects of the local GUI 902 and 906 include an IPaddress field, a LOAD button, a BACK button, a FORWARD button, a MOUSEbutton, and a GEAR button for remotely controlling a web browserapplication 327 running on the server 204.

Referring to the figures discussed above, various operations of themodules in a 204 sever and in a client device 202 are further describedbelow in accordance with one aspect of the disclosure.

Now referring to FIGS. 5B and 7, the client device 202 may furthercomprise an input device 560 and a user input module 550. In oneexample, the input device 560 comprises a touch screen overlaying thedisplay 560. In this aspect, the user may enter user inputs within adisplay area corresponding to the local GUI 702 and 706 by tapping on adesired graphical control object using a finger or stylus. The userinput module 550 may send user inputs to the local GUI 702 and 706 forremotely controlling the application 327 to the local view module 540.For the example of a touch screen, when the user touches a display areacorresponding to the local GUI 702 and 706, the user input module 550directs the corresponding user inputs to the local view module 540. Theuser inputs may comprise coordinates of the location where the usertouched the display 560.

The user may also enter user inputs within a remote application view 704of a remote application 327 on the display 560. For the example of atouch screen, the user may enter user inputs by moving a pointer (notshown) within a display area corresponding to the remote applicationview 704 of the remote application 327. In this aspect, the user inputmodule 550 directs user inputs within the remote view 704 to the remoteview module 535.

In one aspect of the disclosure, the local view module 540 may beconfigured to generate a command message based on user inputs to thelocal GUI 702 and 706 for remotely controlling the application 327 andsend the command message to the server 204. For the example of a touchscreen, the local view module 540 interprets user inputs to the localGUI 702 or 706 and then translates them into corresponding commandmessages. In this example, a user input may comprise coordinates of alocation where the user touches the local GUI 702 or 706 on the display560 or coordinates of a pointer within the local GUI 702 or 706controlled by the user, for example, using a pointer device. The localview module 540 can determine which graphical control object is selectedby the user based on the graphical control object in the local GUI 702and 706 corresponding to the coordinates of the user inputs. The localview module 540 determines the function associated with the selectedgraphical control object and generates a command message for the server204 with a command to perform the function. For example, if the usertouches the local GUI 702 or 706 (e.g., using a finger) at a locationcorresponding to a refresh button in the local GUI 702 and 706, then thelocal view module 540 generates a command message for the server 204 torefresh the web page.

In one aspect, the agent module 322 on the server 204 receives thecommand message and issues a command to the application control module325 to control the application 327 based on the command message. Forexample, if the command message is to refresh the web page, then theagent module 322 instructs the application control module 325 to reloadthe web page on the web browser application 327. Thus, the local viewmodule 540 generates command messages based on user inputs to the localGUI and sends the command messages to the agent module 322 on the server204, and the agent module 322 controls the remote application based onthe received command messages using the application control module 325.

In one aspect, when a local view module 540 receives one or more userinputs directed to, or placed into, a local GUI (e.g., 702 or 706), thelocal view module 540 may interpret the one or more user inputs,determine (or identify) function(s) corresponding to the one or moreuser inputs, and generate one or more command messages corresponding tothe function(s) by, for example, translating the one or more user inputsinto the command message(s). Determining the function of a user inputmay, for example, include identifying the type of user input (e.g., atext entry, a button selection, a menu selection) and/or determining afunction corresponding to the user input (e.g., a text entry for an IPaddress field, a selection of a BACK button, etc.). For example, whenthe user input comprises a selection of a button (e.g., refresh button)in the local GUI (e.g., based on coordinates of a user's touch on thedisplay 560), the local view module 540 may determine the function(e.g., refresh current web page) associated with the selected button. Inone aspect, these functions are predetermined. The local view module 540then generates a command message based on the determined function. Someexamples of command messages may include, but are not limited to, acommand to load a web page from an IP address, display a previouslyviewed web page, display a later viewed web page, refresh or reload acurrent web page, stop loading of a webpage, zoom in or out, switchapplications, open bookmarks or history (e.g., for a web browserapplication), and other commands. A command message may comprise acommand packet that is transmitted from the client device 202 to theserver 204 using various protocols, compressions and encryption schemes.The server 204 may also send commands and status information to theclient device 202. For example, the server 204 may send the clientdevice 202 a web page loading status, a redirected URL or keyboardstate.

In one aspect, the remote view module 535 may be configured to send userinputs received within the remote application view 704 to the server 204via the remote access module 452. The user inputs may include pointerinputs comprising coordinates of pointer movements and clicks (e.g.,mouse clicks). For example, the user may move a pointer (not shown)within the remote application view 704 using a touch screen, a touchpad, a trackball, a mouse or other pointer input device. In thisexample, the coordinates of the pointer movements may be transmitted tothe server 204. The user inputs may also include keyboard inputs. Theuser inputs may enter keyboard inputs using a graphical keyboard (e.g.,820 a) displayed on the display 560, a keypad or other device. Forexample, when the user desires to enter text at the location of apointer or cursor within the remote application view 704, the user maytap on a keyboard icon (e.g., 920) to bring up the graphical keyboard toenter the text. In one aspect, the client device access module 452 maysend user inputs comprising pointer inputs (e.g., coordinates of pointermovements) and keyboard inputs to the server 204 using RDP, ICA or otherremote access protocol.

In one aspect, the server remote access module 334 may receive the userinputs from the client remote access module 452 over the remote accessconnection 525 and sends the user inputs to the application controlmodule 325. The application control module 325 interprets the receiveduser inputs and controls the application 327 accordingly.

In one aspect, when an application 327 updates its display output inresponse to a received command message or user inputs, the remote accessmodule 334 may send updated display output data to the client device202. The client remote access module 452 receives the updated displayoutput data of the remote application 327 and sends the updated displayoutput data to the remote view module 535. The remote view module 535then generates an updated image of the display output of the remoteapplication, which is displayed within the remote application view 704of the display 560.

Aspects of the disclosure allow the user at the client device 202 toremotely view an application running on the server 204 while controllingthe application using a local GUI that can be optimized for the display560 of the client device 202.

An advantage of aspects of the disclosure may be illustrated withreference to FIGS. 9 and 1B. FIG. 9 shows an example of a composite view900 at the client device 202 for a web browser application running onthe server 204, according to an aspect of the disclosure. In thisexample, the GUI of the application 327 running on the server 204 may besimilar to the GUI 122 shown in FIG. 1B, which is designed for a largedisplay. Instead of displaying an image of the remote application's GUI122 running on the server 204, the local view module 540 advantageouslygenerates and displays a local GUI 902 and 906 for controlling theremote application 327. In one aspect, the local GUI 902 and 906 in FIG.9 is not based on the display output of the remote application 327. Thelocal GUI 902 and 906 may be optimized for the display 560 of the clientdevice 202. An image of the display output of the remote web browserapplication 327 is displayed within the remote application view 904allowing the user at the client device 202 to remotely view the webbrowser application 327 while remotely controlling the web browserapplication 327 using the local GUI 902 and 906.

The local GUI 702 and 706 may be designed to have a similar look andfeel of a web browser application that is native to the client device(e.g., smartphone). This allows the user at the client device to controla web browser application 327 running on a server 204 in a similarmanner as a web browser that is native to the client device (e.g.,smartphone), while enjoying the benefits of server-side computing suchas increased computing power and resources available on the server 204.For example, a web browser application 327 running on a server hasaccess to plug-in applications on the server 204 for renderingmulti-media content. As a result, the plug-in applications do not haveto be, for example, loaded onto the client device 202. Another advantageof the local GUI is that it allows a user at the client device 202 touse the application on the server 204 in a user friendly environmentwithout having to extensively rewrite the application port theapplication to the client device.

In an aspect of the disclosure, the local GUI and the remote applicationview 704 on the display 560 can be controlled independently. Forexample, the remote view module 535 may allow the user to adjust thezoom and scroll the image of the remote application 327 within theremote application view 704 while the local view module 540 maintainsthe size and location of the local GUI 702 and 706 on the display 560.Thus, the local GUI 702 and 706 can remain on the display 560 and bereadily accessible to the user while the user manipulates the image ofthe remote application within the remote application view 704.

In an aspect of the disclosure, the agent module 322 may instruct theapplication control module 325 to deactivate the GUI of the application327 so that display output data of the application 327 sent to theclient device 202 does not include the GUI of the application. For theexample of the web browser 120 running on a server in FIG. 1B, the agentmodule 322 may deactivate the GUI 122 so that only the display of theweb content 124 is sent to the client device 202. This allows a user atthe client device 202 to view the web content 124 of the web browser 120running on the server 204 while controlling the web browser 120 with thelocal GUI 702 and 706 instead of the remote application's GUI 122running on the server 204.

The local GUI at the client device 202 is not limited to the example ofa web browser application and may be used to remotely control many otherapplications on the server 204. Examples of applications that may becontrolled by local GUIs include word processing applications, spreadsheet applications, multi-media player applications, electronic mailapplications, and other applications. For the example of a wordprocessing application, the local GUI may include graphical controlobjects for print, edit, and formatting functions. For a multi-mediaplayer, the local GUI may include graphical control objects for play,stop and rewind functions.

In one aspect, the client device 202 may have a plurality of local GUIsfor controlling different applications. In this aspect, when the user atthe client device 202 remotely launches an application on the server204, the agent module 322 may determine what type of application waslaunched. The agent module 322 may then send a message to the clientdevice 202 via the server remote access module 334 indicating the typeof application that was launched on the server 204. The client remoteaccess module 452 sends the received message to the local view module540. The local view module 540 may then select a local GUI from aplurality of local GUIs that matches the type of application indicatedby the received message. The agent 322 may also send a configurationfile to the client device 202 specifying the local GUI. Thus, the localview module 540 can change the local GUI to adapt to changes in theapplication running on the server 204.

FIG. 10 is a conceptual block diagram illustrating a server remoteaccess module 334 and a client remote access module 452 according to anaspect of the disclosure. In one aspect, a server remote access module334 comprises first and second server remote connection modules 1020 aand 1020 b and the client remote access module 452 comprises first andsecond client remote connection modules 1030 a and 1030 b. The firstserver remote connection module 1020 a and the first client remoteconnection module 1030 a communicate over connection 1025 a. The secondserver remote connection module 1020 b and the second client remoteconnection module 1030 b communicate over connection 1025 b. Theconnections 1025 a and 1025 b may be established between the server 204and the client device 202 over a network 206 using different sockets. Inthis aspect, different types of information may be communicated over thedifferent connections 1025 a and 1025 b.

In one aspect of the disclosure, the first server remote connectionmodule 1020 a and the first client remote connection module 1030 a areimplemented using a remote access application (e.g., RDP application)for communicating display output data and user inputs between the server204 and the client device 202. The connection 1025 a may comprise aplurality of virtual channels with different virtual channels carryingdisplay output data (e.g., images or audio output of a remoteapplication) and user inputs (e.g., pointer and keyboard inputs from auser at a client device) between the server 204 and the client device202.

The second server remote connection module 1020 b and the second clientremote connection module 1030 b may be configured to communicate commandmessages over the connection 1025 b. In this aspect, the second clientremote connection module 1030 b sends command messages from the localview module 540 to the server 204 over the connection 1025 b. The secondclient remote connection module 1020 b communicates the received commandmessages to the agent module 322. In this aspect, the agent module 322may manage the connection 1025 b. In one aspect, the second remoteconnection module 1020 b may be integrated with the agent module 322.Thus, in this aspect, the command messages are communicated over adifferent connection than the display data and user inputs. An advantageof this aspect is that the command messages may be communicated using adifferent protocol than the display data and user inputs. For example,the display output data and user inputs may be communicated using aremote access protocol (e.g., RDP) while the command messages arecommunicated using a different protocol. For example, the commandmessages may be communicated using a protocol having differentencryption, compression and/or security features than the remote accessprotocol.

FIG. 11 depicts an example of the connection 1025 a comprising multiplevirtual channels 1120 a to 1120 c and the connection 1025 b. In thisexample, the connection 1025 a may be based on a remote access protocol(e.g., RDP) for communicating display output data and user inputs ondifferent virtual channels 1120 a to 1120 c. The connection 1025 b isused to communicate command messages. Alternatively, the commandmessages may be sent over the connection 1025 a, in which case thesecond remote connection modules 1020 b and 1030 b may be omitted. Forexample, the command messages may be sent on a virtual channel of theconnection 1025 a established for command messages. In an aspect, acommand message may include a flag or other indicator identifying themessage as a command message.

FIGS. 10 and 11 illustrate merely examples of communications between aserver and a client device. A server may include one or more remoteconnection modules (e.g., one, two, three or more), and a client deviceinclude one or more remote connection modules (e.g., one, two, three ormore).

FIG. 12 is a conceptual block diagram illustrating an example of asystem.

A system 1200 may be, for example, a client device or a server. Thesystem 1200 includes a processing system 1202. The processing system1202 is capable of communication with a receiver 1206 and a transmitter1209 through a bus 1204 or other structures or devices. It should beunderstood that communication means other than busses can be utilizedwith the disclosed configurations. The processing system 1202 cangenerate audio, video, multimedia, and/or other types of data to beprovided to the transmitter 1209 for communication. In addition, audio,video, multimedia, and/or other types of data can be received at thereceiver 1206, and processed by the processing system 1202.

The processing system 1202 may include a general-purpose processor or aspecific-purpose processor for executing instructions and may furtherinclude a machine-readable medium 1219, such as a volatile ornon-volatile memory, for storing data and/or instructions for softwareprograms. The instructions, which may be stored in a machine-readablemedium 1210 and/or 1219, may be executed by the processing system 1202to control and manage access to the various networks, as well as provideother communication and processing functions. The instructions may alsoinclude instructions executed by the processing system 1202 for varioususer interface devices, such as a display 1212 and a keypad 1214. Theprocessing system 1202 may include an input port 1222 and an output port1224. Each of the input port 1222 and the output port 1224 may includeone or more ports. The input port 1222 and the output port 1224 may bethe same port (e.g., a bi-directional port) or may be different ports.

The processing system 1202 may be implemented using software, hardware,or a combination of both. By way of example, the processing system 102may be implemented with one or more processors. A processor may be ageneral-purpose microprocessor, a microcontroller, a Digital SignalProcessor (DSP), an Application Specific Integrated Circuit (ASIC), aField Programmable Gate Array (FPGA), a Programmable Logic Device (PLD),a controller, a state machine, gated logic, discrete hardwarecomponents, or any other suitable device that can perform calculationsor other manipulations of information.

A machine-readable medium can be one or more machine-readable media.Software shall be construed broadly to mean instructions, data, or anycombination thereof, whether referred to as software, firmware,middleware, microcode, hardware description language, or otherwise.Instructions may include code (e.g., in source code format, binary codeformat, executable code format, or any other suitable format of code).

Machine-readable media (e.g., 1219) may include storage integrated intoa processing system, such as might be the case with an ASIC.Machine-readable media (e.g., 1210) may also include storage external toa processing system, such as a Random Access Memory (RAM), a flashmemory, a Read Only Memory (ROM), a Programmable Read-Only Memory(PROM), an Erasable PROM (EPROM), registers, a hard disk, a removabledisk, a CD-ROM, a DVD, or any other suitable storage device. Inaddition, machine-readable media may include a transmission line or acarrier wave that encodes a data signal. Those skilled in the art willrecognize how best to implement the described functionality for theprocessing system 1202. According to one aspect of the disclosure, amachine-readable medium is a computer-readable medium encoded or storedwith instructions and is a computing element, which defines structuraland functional interrelationships between the instructions and the restof the system, which permit the instructions' functionality to berealized. Instructions may be executable, for example, by a clientdevice or server or by a processing system of a client device or server.Instructions can be, for example, a computer program including code.

An interface 1216 may be any type of interface and may reside betweenany of the components shown in FIG. 12. An interface 1216 may also be,for example, an interface to the outside world (e.g., an Internetnetwork interface). A transceiver block 1207 may represent one or moretransceivers, and each transceiver may include a receiver 1206 and atransmitter 1209. A functionality implemented in a processing system1202 may be implemented in a portion of a receiver 1206, a portion of atransmitter 1209, a portion of a machine-readable medium 1210, a portionof a display 1212, a portion of a keypad 1214, or a portion of aninterface 1216, and vice versa.

Referring back to FIGS. 2 through 5B and 8B, in one aspect of thedisclosure, the agent module 322 may be configured to receivenotification of events that occur at the server 204 from the OS module324. For example, the OS module 324 may report certain events to theagent module 322 including when the client device 202 has logged ontothe server 204, when an application is launched on the desktop, when adialog box or new window is opened, when a textbox field is clicked,pointer movements, clicks (e.g., mouse clicks), and/or other events. Toreceive notification of events, the agent module 322 may register as anevents handler with the OS module 324 to receive reports of certainevents.

In one aspect of the disclosure, the agent module 322 may determine anaction to be taken at the client device 202 based on the reported event.The agent module 322 may then generate a message based on the determinedaction and send the message to the client device 202 (e.g., via theserver remote access module 334). For example, when the event is a clickon a textbox, the agent module 322 may determine that a graphicalkeyboard is to be activated on the client device 202 and send a messageto the client device 202 to activate the graphical keyboard 820 a. Atthe client device 202, the local view module 540 receives the messagefrom the agent module 322 and performs an action based on the receivedmessage, as discussed further below. In the above example, the localview module 540 may bring up the graphical keyboard 820 a on the display560 in response to the message.

Examples of events that may be reported to the agent module 322 andmessages generated by the agent module 322 for the client device 202based on the reported events are discussed further below.

In one aspect of the disclosure, the OS module 324 may report a click ona textbox to the agent module 322. This may occur, for example, whenuser inputs (e.g., pointer movements and a click) received from theclient device 202 result in a textbox of the application 327 beingclicked on. When this event occurs, the agent module 322 may determinethat a graphical keyboard is to be activated on the client device 202for the user at the client device 202 to enter text. The agent module322 may then generate a message for the client device 202 including acommand to bring up the graphical keyboard 820 a on the display 560 ofthe client device 202, and send the message to the client device 202(e.g., via the server remote access module 334).

On the client side, the client remote access module 452 receives themessage and sends the received message to the local view module 540.Upon reading the received message, the local view module 540 brings upthe graphical keyboard 820 a on the display 560 of the client device 202to allow the user at the client device 202 to enter text. An example ofthis is illustrated in FIGS. 14A and 14B.

FIG. 14A shows a display 560 at the client device 202, in which theremote application view 704 displays a web page of a remote web browserapplication 327. The web page includes a textbox 1310 (e.g., searchtextbox). In this example, the user at the client device 202 inputs userinputs (e.g., pointer movements and clicks) using a touch screen orother pointer input device at the client device 202. The remote viewmodule 535 sends the user inputs to the server 202 (e.g., via the clientremote access module 452). On the server side, the application controlmodule 325 controls the web browser application 327 based on thereceived user inputs. Updated display output data of the web browserapplication 327 in response to the user inputs are sent to the clientdevice 202. On the client side, the remote view module 535 updates theimage in the remote application view 704 based on the received updateddisplay output data.

On the server side, if a received pointer movement and click result in atextbox being click on, then the OS module 324 reports this event to theagent module 322. The agent module 322 determines that a graphicalkeyboard 802 a is to be activated at the client device 202 and generatesa message for the client device 202 including a command to activate thegraphical keyboard 820 a. The agent module 322 then sends the message tothe client device 202. On the client side, the local view module 540receives the message and brings up the graphical keyboard 820 a on thedisplay 560 in response to the message. FIG. 14B shows the display 560at the client device 202, in which the graphical keyboard 820 a isbrought up on the display 560 after the textbox 1310 has been click on.An I-beam 1415 or other type of cursor may appear in the textbox 1310 toindicate that text will be entered into the textbox 1310.

As the user at the client device 202 inputs text using the graphicalkeyboard 820 a, the text inputs to the graphical keyboard 820 a may bedirected to the remote view module 535, for example, as if the user hadentered the text using a physical keyboard. The remote view module 535may then send the user text inputs to the server 204 (e.g., using aRDP/ICA or other remote access protocol). On the server side, theapplication control module 325 enters the received text into thetextbox. Updated display output data showing the inputted text in thetextbox is then sent back to the client device 202. In this aspect, whenthe user at the client device 202 hits the enter button on the graphicalkeyboard 820 a after entering text, the local view module 540 mayautomatically deactivate the graphical keyboard 820 a.

An advantage of this aspect is that the graphical keyboard 820 a isautomatically brought up on the display 560 when the textbox 1310 isclicked on without requiring the user at the client device 202 to hitthe keyboard button 820. This is not possible in other approaches inwhich only display output data of the remote application 327 is sent tothe client device 202. In these approaches, the client device 202 simplydisplays the display output of the remote application 327 and receivesno information about when a textbox of the remote application is clickedon. By contrast, in the above aspect, the agent module 322 determineswhen the textbox has been clicked on and sends a message to the clientdevice 202 to bring up the graphical keyboard 820 a.

In one aspect, the local view module 540 keeps track of the area of thedisplay output of the remote application 327 that is currently displayedin the remote application view 704. FIG. 15A shows an exemplary area1510 of the display output 1520 that may be displayed in the remoteapplication view 704. In this example, the total area of the displayoutput 1520 may represent what a user at the server 204 might see on alarge display at the server 204. In this example, the server 202 maysend the entire display output of the remote application 327 to theclient device 204, and the client device 204 may store the entiredisplay output in a machine-readable medium (e.g., cache memory).

The local view module 540 may then allow the user to manipulate the area1510 of the display output that is displayed in the remote applicationview 704. For example, the local view module 540 may allow the user tozoom in by moving two fingers away from each other on a touch screen 565starting at a desired zoom in location. The local view module 540 mayalso include zoom in and zoom out buttons on the local GUI 702 and 706.The local view module 540 may also display scrollbars on the display 560that allow the user to scroll (up, down, left, right) the area 1510 ofthe display output 1520 displayed in the remote application view 704.FIG. 15B shows an example in which the area 1510 of the display output1520 displayed in the remote application view 704 has been scrolled tothe right. In this aspect, the local view module 540 may send a commandto the remote view module 535 specifying which area 1510 of the displayoutput 1520 to display in the remote application view 704 based oninputs from the user. The command may include coordinates and dimensionsof the area 1510 to be displayed in the remote application view 704.

As discussed further below, the local view module 540 may keep track ofthe area 1510 of the display output 1520 displayed in the remoteapplication view 704 to determine whether the user at the client device202 can view certain events (e.g., a dialog box popup) communicated tothe local view module 540 by the agent module 322.

In one aspect of the disclosure, the OS module 324 may report to theagent module 322 when a dialog box and/or window opens as a result ofuser inputs (e.g., pointer movements and clicks) received from theclient device 202. For example, a dialog box may open when a pointer ismoved over an object (e.g., icon) and/or when a object is clicked on.The dialog box may display information to the user and/or requestinformation from the user. For an example of a web browser application327, a new browser window may open when the user clicks on a link.

When the OS module 324 reports that a dialog box and/or window hasopened, the agent module 322 may determine that the dialog box and/orwindow is to be displayed to the user at the client device 202. In thisaspect, the OS module 324 may also communicate the coordinates anddimensions of the dialog box or window to the agent module 322. Theagent module 322 may then generate a message for the client device 202including the coordinates and dimensions of the dialog box or window.The agent module 322 may then send the message to the client device 202(e.g., via the server remote access module 334).

On the client side, the client remote access module 452 receives themessage and sends the received message to the local view module 540.Upon reading the received message, the local view module 540 maydetermine whether the newly opened dialog box or window is currentlyshown in the remote application view 704. The local view module 540 maydo this, for example, by comparing the coordinates and dimensions of thedialog box or window with the coordinates and dimensions of the area1510 of the display output 1520 displayed in the remote application view704. If the dialog box or window is not currently shown in the remoteapplication view 704, then the local view module 540 may command theremote view module 535 to scroll and/or zoom out the remote applicationview 704 by a certain amount so that the dialog box or window is shownin the remote application view 704. The local view module 540 maydetermine the amount that the remote application view 704 needs to bescrolled and/or zoomed out based on differences between the coordinatesand/or dimensions of the dialog box or window and the coordinates and/ordimensions of the area 1510 of the remote application view 704. Anexample of this is shown in FIGS. 16A and 16B.

FIG. 16A shows an example in which a dialog box or window 1610 opensoutside the area 1510 of the remote application view 704. As a result,the user at the client device 202 cannot see the dialog box or window1610. For example, the dialog box 1610 may require certain information(e.g., billing information) from the user to continue with a transaction(e.g., purchase). In this example, if the user cannot see the dialog box1610 at the client device 204, then the user may be unaware thatinformation is being requested.

In this example, the agent module 322 may send the coordinates anddimensions of the dialog box 1610 to the local view module 540 on theclient side. The local view module 540 may then determine that thedialog box 1610 is not displayed in the remote application view 704.After making this determination, the local view module 540 may commandthe remote view module 535 to scroll the remote application view 704 sothat the dialog box 1610 is shown in the remote application view 704.FIG. 16B shows an example in which the area 1510 of the remoteapplication view 704 has been scrolled to the right so that the dialogbox 1610 is shown in the remote application view 704. The local viewmodule 540 may also command the remote view module 535 to zoom out theremote application view 704 so that the dialog box 1610 is shown in theremote application view 704. This advantageously allows the user at theclient 204 to view the dialog box 1610 in order to receive informationin the dialog box 1610 and/or enter information requested by the dialogbox 1610.

In one aspect, the local view module 540 may notify the user at theclient device 202 when a dialog box or window 1610 opens outside thearea 1510 of the remote application view 704. In this aspect, the localview module 540 may display an arrow or other indicator on the display560 indicating the direction of the dialog box or window 1610. The usermay then choose whether to scroll the remote view application 704 in theindicated direction to view the dialog box or window 1610.

In one aspect, the OS module 324 may also report to the agent module 322when the dialog box or window closes. For example, a dialog box mayclose when the user has entered information requested by the dialog box.In this aspect, the agent module 322 may send a message to the localview module 540 indicating that the dialog box or window has closed. Thelocal view module 540 may then command the remote view module 535 toreturn the area 1510 of the remote application view 704 back to itsoriginal location before the dialog box or window was opened. In otherwords, the local view module 540 may undo the scrolling and/or zoomingwhen the dialog box or window closes.

In one aspect, the OS module 324 may also report to the agent module 322when a new web page is loaded in a web browser application 327. Theagent module 322 may then send a message to the local view module 540informing the local view module 540 that a new web page has been loaded.The local view module 540 may then command the remote view module 535 tozoom out the remote application view 704 to show substantially theentire display output 1520. This aspect automatically gives the user atthe client device 202 an overview of content in the newly loaded webpage. The user may then decide to zoom and/or scroll the remoteapplication view 704 to focus on a desired portion of the new web page.

In one aspect, the OS module 324 may report the current coordinates(e.g., x-y coordinates) of the pointer to the agent module 322. Thecoordinates of the pointer may refer to coordinates of a pointer movedby the user at the client device 202 using a pointer input device orcoordinates of a location where the user touches a display 560 at theclient device 202 (e.g., using a touch screen overlaying the display560). The agent module 322 may use the current coordinates of thepointer to determine whether the pointer is on an object (e.g., picture)in the application 327. The agent module 322 may do this, for example,by determining the coordinates and dimensions of objects in theapplication 327 and comparing the coordinates and dimensions of theobjects with the reported coordinates of the pointer.

For an example of a web browser application 327, the agent module 322may determine the coordinates (e.g., x-y coordinates) and dimensions(e.g., width and height) of objects (e.g., pictures, icons, animations,banners, etc.) in a web page by analyzing markup language code (e.g.,HTML, XML, etc.) for the web page. The agent module 322 may receive themarkup language code from the application control module 325. The markuplanguage code for the web page may identify objects in the web page andtheir locations in the web page. The markup language code may alsoprovide attributes for each object including object type (e.g., picture)and dimensions (e.g., height and width) of the object. The agent module322 may use this information to determine the types of objects in theweb page and the coordinates and dimensions of the objects in the webpage. The agent module 322 may make this determination each time a newweb page is loaded into the web browser application 327.

In this aspect, when the agent module 322 determines that the pointer ison an object in the application, then the agent module 322 may determinethat substantially the entire object is to be displayed at the clientdevice 202. The agent module 322 may then send a message to the clientdevice 202 including information of the object. The information mayinclude object type (e.g., picture) and the coordinates and dimensionsof the object. On the client side, the local view module 540 may receivethe coordinates and dimensions of the object and use the receivedcoordinates and dimensions of the object to determine whethersubstantially the entire object is shown in the remote application view704. The local view module 540 may do this, for example, by comparingthe coordinates and dimensions of the object with the coordinates anddimensions of the area 1510 of the remote application view 704.

If the local view module 540 determines that substantially the entireobject is not shown in the remote application view 704, then the localview module 540 may command the remote view module 535 to scroll and/orzoom the remote application view module 535 by a certain amount to showsubstantially the entire object. The local view module 540 may determinethe amount based on differences between the coordinates and dimensionsof the object and the coordinates and dimensions of the area 1510 of theremote application view 704. An example of this is illustrated in FIGS.17 and 18.

FIG. 17 shows an example of a display 560 at the client device 202, inwhich the remote application view 704 displays a web page on the remoteweb browser application 327. The web page includes an object 1710 (e.g.,a picture). In this example, only a small portion of the object 1720 isshown in the remote application view 704 and a pointer 1705 is placed onthe object 1710. FIG. 19A shows the portions of the object 1710 that arewithin and outside the area 1510 of the remote application view 704.

On the server side, the agent module 322 determines that the pointer1705 is on the object 1720 based on the coordinates of the pointerreported from the OS module 324 and the coordinates and dimensions ofobjects in the web page. The agent module 322 then generates a messagefor the client device 202 including the coordinates and dimensions ofthe object 1710, and sends the message to the client device 202.

On the client side, the local view module 540 receives the message anddetermines whether substantially the entire object 1710 is shown in theremote application view 704. In this example, the local view module 540determines that only a small portion of the object 1710 is shown in theremote application view 704. The local view module 540 then commands theremote view module 535 to scroll the remote application view 704 to theleft so that substantially the entire object 1710 is shown the remoteapplication view 704. FIG. 18 shows an example of the display 560 afterthe remote application view 704 has been scrolled to the left to showthe entire object 1710 in the remote application view 704. FIG. 19Bshows that, after scrolling to the left, the entire object 1710 iswithin the area 1510 of the remote application view 704.

In one aspect, the local view module 540 may also zoom in on the object1710. For example, the local view module 540 may also command the remoteview module 535 to zoom in on the object 1710 in the remote applicationview 704. For example, the local view module 540 may center the object1710 in the remote application view 704 and maximize the size of theobject 1710 in the remote application view 704.

An advantage of this aspect is that, when the user shows an interest inan object by moving a pointer to the object or touching the object usinga touch screen, substantially the entire object is automatically shownin the remote application view 704. Thus, the user does not have toscroll and/or zoom out to see the rest of the object of interest.

In another example, the object 1710 may be a video. In this example, thevideo file for the object 1710 may be downloaded from a uniform resourcelocator (URL) and played using a media player application. The URL forthe video file may be included in markup language code for a web page orother source.

In this example, the agent module 322 on the server side may determinewhen the pointer is on the object 1710 and/or clicks on the object 1710based on coordinates of the pointer and/or pointer clicks reported fromthe OS module 324. The agent module 322 may then generate a messageincluding the URL for the video file, and send the message to the clientdevice 202.

On the client side, the local view module 540 receives the message,retrieves the URL for the video file from the message, and downloads thevideo file onto the client device 202 from the URL. In this example, theclient remote access module 452 may establish a connection with theInternet or other network, and the local view module 540 may downloadthe video file from the URL through the connection. After the video fileis downloaded or while the video file is downloading, the local viewmodule 540 may launch a local media player application on the clientdevice 202 and play the video file using the local media playerapplication. The local view module 540 may then display the output ofthe local media player application on the display 650. For example, thelocal view module 540 may display the output of the local view module540 over the remote application view 704.

A process for establishing a remote session on the server 204 for a userat the client device 202 will now be discussed according to an aspect ofthe disclosure. When the user at the client device 202 desires toremotely access a remote server, the local view module 540 may display aplurality of remote servers from which the user can select on thedisplay 560. For each available remote server, the client device 202 maystore an address (e.g., IP address) and user credentials (e.g., loginname and password) for the server. The user may then select one of theremote servers on the display 560, for example, using a touch screenoverlaying the display 560.

When the user selects a remote server, the client remote access module452 may initiate a remote access connection 525 with the server remoteaccess module 334 of the server 204. To do this, the client remoteaccess module 452 may send a request for remote access to the server 204(e.g., over a TCP/IP network connection using an IP address of theserver 204). The request may include user credentials (e.g., login nameand password). If the user credentials are accepted, then the serverremote access module 334 of the server 204 establishes a remote sessionfor the client device 202, providing the client device 202 with remoteaccess to the server 204.

In one aspect, the OS module 324 may notify the agent module 322 that aremote session has been established for the client device 202. The agentmodule 322 may then send a message to the client device 202, via theserver remote access module 334, informing the client device 202 thatthe agent module 322 is active and ready to accept command messages fromthe client device 202. On the client side, the client remote accessmodule 452 may send the message to the local view module 540. Uponreceiving the message, the local view module 540 may send anacknowledgement to the server 204. On the server side, the server remoteaccess module 334 may send the acknowledgement to the agent module 322.This process may be used to initialize communication between the agentmodule 322 and the local view module 540.

After initializing communication with the agent module 322, the localview module 540 may display a plurality of applications that areavailable on the server 204. The local view module 540 may display alist of the available applications in a menu on the display 560 and theuser may select a desired one of the applications, for example, using atouch screen. FIG. 20 shows an example of a local GUI 2006 listingapplications that are available on the server 204. The local GUI 2006 islocally generated and not generated from display output data from theserver 204.

When the user selects an application, the local view module 540 maygenerate a command message identifying the selected application and sendthe command message to the server 204 via the client remote accessmodule 452. On the server side, the server remote access module 334 maysend the command to the agent module 322. The agent module 322 may thenissue a command to the OS module 324 to launch the identifiedapplication 327 on the server 204. The agent module 322 may also commandthe application control module 325 to maximize the size of the displayoutput of the application 327.

The server remote access module 334 may then send display output data ofthe application 327 to the client device 202. On the client side, thelocal view module 540 may display a local GUI 702 and 706 for theselected application on the display 560. The local view module 540 mayalso translate user inputs to the local GUI 702 and 706 intocorresponding command messages and send the command message to the agentmodule 322, as discussed above.

In one aspect of the disclosure, the desktop module 328 may generate adesktop on the server 204. The desktop may include icons of availableapplications that can be selected and launched on the server 204. In oneaspect, the desktop module 328 may be integrated with the operatingsystem 330 (e.g., Windows-based operating system).

In this aspect, the server remote access module 334 may send displayoutput data of the desktop to the client device 202. On the client side,the remote view module 535 may display the remote desktop in the remoteapplication view 704 on the display 560. FIG. 21 shows an example of aremote desktop 2110 displayed in the remote application view 704 at theclient device 202. The remote desktop 2110 includes icons 2115 ofapplications that can be launched on the server 204 and a taskbar 2120.

In this aspect, the user at the client device 202 may select anapplication on the remote desktop displayed in the remote applicationview 704, for example, by moving a pointer 2125 to the icon 2115 of adesired application and clicking on the icon 2115 using a pointer inputdevice. The remote view module 535 may then send the entered user inputs(e.g., pointer movements and clicks) to the server 202. On the serverside, the server remote access module 334 may input the received userinputs to the desktop module 328. When received user inputs to thedesktop result in an application being launched on the server 204, theOS module 324 may inform the agent module 322. The agent module 322 maythen generate a message identifying the launched application and sendthe message to the client device 204 (e.g., via the server remote accessmodule 334).

On the client side, the client remote access module 452 may send themessage to the local view module 540. The local view module 540 may thendisplay a local GUI 702 and 706 for the identified application on thedisplay 560. For example, the client device 202 may store configurationfiles for a plurality of local GUIs for different applications. Thelocal view module 540 may then retrieve the configuration file 451 forthe local GUI 702 and 706 corresponding to the application identified inthe received message and generate the local GUI based on the retrievedconfiguration file 451. The local view module 540 may then display thelocal GUI 702 and 706 on the display 560 to allow the user to remotelycontrol the application 327 on the server 204, as discussed above.

In this aspect, the local view module 540 may also generate a local GUIfor the user at the client device 202 to control the remote desktop atthe server 204. For example, the local GUI for the remote desktop mayallow the user to select a background for the desktop, hide a taskbar ofthe desktop, and other functions. In another example, the local GUI forthe remote desktop may allow the user to minimize, resize, hide and/ormove a window for an application on the remote desktop. When the userenters user inputs to the local GUI for the remote desktop, then thelocal view module 540 may translate the user inputs into a commandmessage for the remote desktop. For example, the local view module 540may translate user inputs to a graphical control object (e.g., hidetaskbar button) on the local GUI into a command message to hide thetaskbar. The local view module 540 may send the command message to theserver 204 via the client remote access module 452. On the server side,the server remote access module 334 may send the command message to theagent module 322. The agent module 322 may then issue a command to thedesktop module 328 based on the received command message. For example,when the received command message is to hide the taskbar, then the agentmodule 322 may issue a command to the desktop module 328 to hide thetaskbar.

FIG. 22 is a conceptual block diagram of the agent module 322 accordingto an aspect of the disclosure. In this aspect, the agent module 322comprising a master agent module 2222 and a plurality of slave agentmodules 2224-1 to 2224-3. The agent module 322 may include any number ofslave agent modules 2224-1 to 2224-3. In this aspect, each slave moduleagent 2224-1 to 2224-3 may be configured to control a particularapplication module 326-1 to 326-3 based on command messages from theclient device 202. The master agent module 2222 may be configured toreceive a command message from the client device 202 (via the serverremote access module 334) and direct the command message to one of theslave agent modules 2224-1 to 2224-3.

In one aspect, there may be a number of applications 327-1 to 327-3running on the server 204, in which each of the correspondingapplication modules 326-1 to 326-3 is controlled by one of the slaveagent modules 2224-1 to 2224-2. In this aspect, the OS module 324 maynotify the master agent module 2222 which one of the applications 327-1to 327-3 is currently in focus on the server 204. For example, anapplication 327-2 to 327-3 may be in focus when an active pointer,cursor or I-beam appears in the window of the application 327-2 to327-3. The master agent module 2222 may then direct command messagesfrom the client device 202 to the slave agent module 2224-1 to 2224-3corresponding to the application 327-1 to 327-3 currently in focus.

In one aspect, the user the client device 202 may indicate which one ofa plurality of applications 327-1 to 327-3 currently running on theserver 202 he or she desires to access at a given time. For example, thelocal view module 540 may display a taskbar in the local GUI 702 and 704indicating which application are currently running on the server 204. Inthis aspect, the OS module 324 may notify the master agent module 2222which applications are running on the server 204 and the master agentmay send a message to the client device 202 identifying theseapplications. The local view module 540 may then generate a localtaskbar identifying these applications based on the message. When theuser selects an application from the taskbar (e.g., using a touchscreen), the local view module 540 may send a command message to theserver 204 identifying the application.

On the server side, the server remote access module 334 may send thecommand message to the master agent module 2222. The master agent module2222 may then send a command to the desktop module 328 to maximize theselected application and minimize or hide the other applications runningon the server 204 so that the display output from the sever 204 to theclient device 202 only shows the selected application. The master agentmodule 2222 may then direct subsequent command messages from the clientdevice 202 to the slave agent module 2224-1 to 224-3 corresponding tothe selected application 327-1 to 327-3.

The subject technology is illustrated, for example, according to variousaspects described below. Numbered clauses are provided below forconvenience. These are provided as examples, and do not limit thesubject technology.

1. A system for rendering at a local client side a composite viewincluding a local graphical user interface and a remote application viewassociated with a remote application running at a remote server,comprising:

a local view module configured to generate a local graphical userinterface (GUI) for controlling the remote application remotely from thelocal client side, configured to receive one or more user inputs to thelocal GUI, configured to determine a function corresponding to the oneor more user inputs, configured to generate a command message based onthe determined function, and configured to direct the command message tothe remote server;

a remote view module configured to generate a remote application view ofthe remote application based on display output data of the remoteapplication received at the local client side from the remote server;and

a display module configured to provide, to a local display at the localclient side, a composite view comprising the local GUI and the remoteapplication view of the remote application.

2. The system of clause 1, wherein the local view module is configuredto receive a message identifying a type of application running on theremote server and to select the local GUI from a plurality of local GUIsbased on the received message.

3. The system of clause 1, wherein the local view module is configuredto be controlled dynamically by the remote server to generate the localGUI based on the application running on the remote server.

4. The system of clause 1, wherein the remote view module is configuredto direct user inputs within the remote application view to the remoteserver.

5. The system of clause 4, wherein the user inputs from the remote viewmodule comprise pointer inputs, keyboard inputs or finger inputs (e.g.,gestures).

6. The system of clause 1, wherein the display output data includes animage or drawing commands of a display output of the remote application.

7. The system of clause 1, wherein the one or more user inputs to thelocal GUI include coordinates of a pointer or a user's touch on thelocal display, and the local view module is configured to determine thefunction based on a graphical control object of the local GUIcorresponding to the coordinates of the pointer or the user's touch onthe local display.

8. The system of clause 1, wherein the local view module is configuredto control a size and a location of the local GUI on the local displayindependently from the remote application view of the remoteapplication.

9. The system of clause 1, wherein the local view module is configuredto direct the command message to the remote server through a remoteaccess module at the local client side.

10. The system of clause 9, wherein the remote access module isconfigured to communicate with the remote server over a first connectionand a second connection via different socket connections, to receive thedisplay output data of the remote application and to send user inputsover the first connection and to send the command message over thesecond connection.

11. The system of clause 9, wherein the remote access module isconfigured to communicate with the remote server over a plurality ofvirtual channels, to send user inputs over a first one of the pluralityof virtual channels and to send the command message over a second one ofthe plurality of virtual channels.

12. A method for rendering at a local client side a composite viewincluding a local graphical user interface and a remote application viewassociated with a remote application running at a remote server,comprising:

generating a local graphical user interface (GUI) for remotelycontrolling the remote application from the local client device (e.g.,1310-A of FIG. 13A);

generating a remote view of the remote application based on displayoutput data of the remote application received at the local clientdevice from the remote server (e.g., 1320-A);

providing, to a local display, a composite view comprising the local GUIand the remote view of the remote application (e.g., 1330-A);

determining a function based on one or more user inputs to the local GUI(e.g., 1340-A);

generating a command message based on the determined function (e.g.,1350-A); and

directing the command message to the remote server (e.g., 1360-A).

13. The method of clause 12, further comprising:

receiving a message identifying a type of application running on theremote server; and

selecting the local GUI from a plurality of local GUIs based on thereceived message.

14. The method of clause 12, further comprising directing user inputswithin the remote application view to the remote server.

15. The method of clause 14, wherein the user inputs from the remoteview module comprise pointer inputs or keyboard inputs.

16. The method of clause 12, wherein the display output data includes animage or drawing commands of a display output of the remote application.

17. The method of clause 12, wherein the one or more user inputs to thelocal GUI include coordinates of a pointer or a user's touch on thelocal display, and the determining the function based on the one or moreuser inputs comprises:

determining the function based on a graphical control object of thelocal GUI corresponding to the coordinates of the pointer or the user'stouch on the local display.

18. The method of clause 12, further comprising controlling a size and alocation of the local GUI on the local display independently from theremote application view of the remote application.

19. The method of clause 12, wherein the directing the command messageto the remote server comprises directing the command message to theremote server through a remote access module at the local client side.

20. The method of clause 19, further comprising:

communicating with the remote server over a first connection and asecond connection via different socket connections; and

sending user inputs over the first connection,

wherein the receiving the display output data comprises receiving thedisplay output data of the remote application over the first connection,and the directing the command message to the remote server comprisessending the command message over the second connection.

21. The method of clause 19, further comprising:

communicating with the remote server over a plurality of virtualchannels; and

sending user inputs over a first one of the plurality of virtualchannels,

wherein the directing the command message to the remote server comprisessending the command message over a second one of the plurality ofvirtual channels.

22. A machine-readable medium encoded with instructions for rendering ata local client side a composite view including a local graphical userinterface and a remote application view associated with a remoteapplication running at a remote server, the instructions comprising codefor:

generating a local graphical user interface (GUI) for remotelycontrolling the remote application from the local client device;

generating a remote view of the remote application based on displayoutput data of the remote application received at the local clientdevice from the remote server;

providing, to a local display, a composite view comprising the local GUIand the remote view of the remote application;

determining a function based on one or more user inputs to the localGUI;

generating a command message based on the determined function; and

directing the command message to the remote server.

23. The machine-readable medium of clause 22, wherein the instructionsfurther comprise code for:

receiving a message identifying a type of application running on theremote server; and

selecting the local GUI from a plurality of local GUIs based on thereceived message.

24. The machine-readable medium of clause 22, wherein the instructionsfurther comprise code for directing user inputs within the remoteapplication view to the remote server.

25. The machine-readable medium of clause 24, wherein the user inputsfrom the remote view module comprise pointer inputs, keyboard inputs, orfinger inputs (e.g., touch inputs).

26. The machine-readable medium of clause 22, wherein the display outputdata includes an image or drawing commands of a display output of theremote application.

27. The machine-readable medium of clause 22, wherein the one or moreuser inputs to the local GUI include coordinates of a pointer or auser's touch on the local display, and the determining the functionbased on the one or more user inputs comprises:

determining the function based on a graphical control object of thelocal GUI corresponding to the coordinates of the pointer or the user'stouch on the local display.

28. The machine-readable medium of clause 22, wherein the instructionsfurther comprise code for controlling a size and a location of the localGUI on the local display independently from the remote application viewof the remote application.

29. The machine-readable medium of clause 22, wherein directing thecommand message to the remote server comprises directing the commandmessage to the remote server through a remote access module at the localclient side.

30. The machine-readable medium of clause 29, wherein the instructionsfurther comprise code for:

communicating with the remote server over a first connection and asecond connection via different socket connections; and

sending user inputs over the first connection,

wherein the receiving the display output data comprises receiving thedisplay output data of the remote application over the first connection,and the directing the command message to the remote server comprisessending the command message over the second connection.

31. The machine-readable medium of clause 29, wherein the instructionsfurther comprise code for:

communicating with the remote server over a plurality of virtualchannels; and

sending user inputs over a first one of the plurality of virtualchannels,

wherein the directing the command message to the remote server comprisessending the command message over a second one of the plurality ofvirtual channels.

32. An apparatus for rendering at a local client side a composite viewincluding a local graphical user interface and a remote application viewassociated with a remote application running at a remote server,comprising:

means for generating a local graphical user interface (GUI) for remotelycontrolling the remote application from the local client device (e.g.,1310-B of FIG. 13B);

means for generating a remote view of the remote application based ondisplay output data of the remote application received at the localclient device from the remote server (e.g., 1320-B);

means for providing, to a local display, a composite view comprising thelocal GUI and the remote view of the remote application (e.g., 1330-B);

means for determining a function based on one or more user inputs to thelocal GUI (e.g., 1340-B);

means for generating a command message based on the determined function(e.g., 1350-B); and

means for directing the command message to the remote server (e.g.,1360-B).

33. The apparatus of clause 32, further comprising:

means for receiving a message identifying a type of application runningon the remote server; and

means for selecting the local GUI from a plurality of local GUIs basedon the received message.

34. The apparatus of clause 32, further comprising means for directinguser inputs within the remote application view to the remote server.

35. The apparatus of clause 34, wherein the user inputs from the remoteview module comprise pointer inputs or keyboard inputs.

36. The apparatus of clause 32, wherein the display output data includesan image or drawing commands of a display output of the remoteapplication.

37. The apparatus of clause 32, wherein the one or more user inputs tothe local GUI include coordinates of a pointer or a user's touch on thelocal display, and the means for determining the function based on theone or more user inputs comprising:

means for determining the function based on a graphical control objectof the local GUI corresponding to the coordinates of the pointer or theuser's touch on the local display.

38. The apparatus of clause 32, further comprising means for controllinga size and a location of the local GUI on the local displayindependently from the remote application view of the remoteapplication.

39. The apparatus of clause 32, wherein the means for directing thecommand message to the remote server comprises means for directing thecommand message to the remote server through a remote access module atthe local client side.

40. The apparatus of clause 39, further comprising:

means for communicating with the remote server over a first connectionand a second connection via different socket connections; and

means for sending user inputs over the first connection,

wherein the means for receiving the display output data comprises meansfor receiving the display output data of the remote application over thefirst connection, and the means for directing the command message to theremote server comprises means for sending the command message over thesecond connection.

41. The apparatus of clause 39, further comprising:

means for communicating with the remote server over a plurality ofvirtual channels; and

means for sending user inputs over a first one of the plurality ofvirtual channels,

wherein the means for directing the command message to the remote servercomprises means for sending the command message over a second one of theplurality of virtual channels.

The subject technology is illustrated, for example, according to variousaspects described below. Numbered clauses are provided below forconvenience. These are provided as examples, and do not limit thesubject technology.

1. A system for communication and for providing, to a remote clientdevice, a message related to an event at a server, in which the event isin response to one or more user inputs received from the remote clientdevice, comprising:

an agent module configured to receive notification of the event at theserver, configured to determine an action, based on the event, to beperformed by the remote client device on a graphical user interface(GUI) at the remote client device or a remote view at the remote clientdevice, configured to generate the message based on the determinedaction, and configured to direct the message to the remote clientdevice,

wherein the GUI is a GUI for controlling an application running on theserver from the remote client device, and the remote view is a view ofthe application running on the server.

2. The system of clause 1, wherein the event comprises a click on atextbox, the action is to activate a graphical keyboard of the GUI atthe remote client device, and the message comprises a command toactivate the graphical keyboard.

3. The system of clause 1, wherein the event comprises a window openingat the server, the action is to display the window within the remoteview at the remote client device, and the message comprises coordinatesand dimensions of the window.

4. The system of clause 1, wherein the event comprises a pointer on anobject of the application at the server, the action is to displaysubstantially the entire object within the remote view at the remoteclient device, and the message comprises coordinates and dimensions ofthe object.

5. The system of clause 4, wherein the object comprises a picture, anicon, an animation, or video.

6. The system of clause 1, wherein the event comprises a pointer clickon a video object, the action is to play a video file associated withthe video object at the remote client device, and the message comprisesa uniform resource locator (URL) for downloading the video file.

7. The system of clause 1, wherein the event comprises a dialog boxopening at the server, the action is to display the dialog box withinthe remote view at the remote client device, and the message comprisescoordinates and dimensions of the dialog box.

8. The system of clause 7, wherein the agent module is configured toreceive notification of when the dialog box closes at the server,configured to generate a second message informing the remote clientdevice that the dialog box has closed, and directing the second messageto the remote client device.

9. The system of clause 1, wherein the agent module is configured toreceive a command message from the remote client device and configuredto control the application on the server based on the received commandmessage.

10. The system of clause 9, wherein the agent module is configured toresize, move or maximize a view of the application on the server basedon the received command message.

11. The system of clause 1, wherein the agent module comprises:

a plurality of slave agent modules, wherein each of the slave agentmodules is configured to control one of a plurality of differentapplications on the server based on command messages from the remoteclient device; and

a master agent module configured to direct command messages, from theremote client device, for one of the plurality of different applicationsto a corresponding one of the plurality of slave agent modules.

12. The system of clause 1, wherein the system is the server comprisinga processing system.

13. A method performed at a server for providing, to a remote clientdevice, a message related to an event at the server, in which the eventis in response to one or more user inputs received from the remoteclient device, comprising:

receiving notification of the event at the server (e.g., 2310-A of FIG.23A);

determining an action, based on the event, to be performed by the remoteclient device on a graphical user interface (GUI) at the remote clientdevice or a remote view at the remote client device (e.g., 2320-A);

generating the message based on the determined action (e.g., 2330-A);and

directing the message to the remote client device (e.g., 2340-A),

wherein the GUI is a GUI for controlling an application running on theserver from the remote client device, and the remote view is a view ofthe application running on the server.

14. The method of clause 13, wherein the event comprises a click on atextbox, the action is to activate a graphical keyboard of the GUI atthe remote client device, and the message comprises a command toactivate the graphical keyboard.

15. The method of clause 13, wherein the event comprises a windowopening at the server, the action is to display the window within theremote view at the remote client device, and the message comprisescoordinates and dimensions of the window.

16. The method of clause 13, wherein the event comprises a pointer on anobject of the application at the server, the action is to displaysubstantially the entire object within the remote view at the remoteclient device, and the message comprises coordinates and dimensions ofthe object.

17. The method of clause 16, wherein the object comprises a picture, anicon, an animation, or video.

18. The method of clause 13, wherein the event comprises a pointer clickon a video object, the action is to play a video file associated withthe video object at the remote client device, and the message comprisesa uniform resource locator (URL) for downloading the video file.

19. The method of clause 13, wherein the event comprises a dialog boxopening at the server, the action is to display the dialog box withinthe remote view at the remote client device, and the message comprisescoordinates and dimensions of the dialog box.

20. The method of clause 19, further comprising:

receiving notification of when the dialog box closes at the server;

generating a second message informing the remote client device that thedialog box has closed; and

directing the second message to the remote client device.

21. The method of clause 13, further comprising:

receiving a command message from the remote client device; and

controlling the application on the server based on the received commandmessage.

22. The method of clause 21, further comprising:

resizing, moving or maximizing a view of the application on the serverbased on the received command message.

23. The method of clause 13, further comprising:

receiving command messages from the remote client devices;

directing the received command messages to one of a plurality ofdifferent applications on the server; and

controlling the one of the plurality of different applications on theserver based on the received command messages.

24. A machine-readable medium encoded with instructions for providing,to a remote client device, a message related to an event at a server, inwhich the event is in response to one or more user inputs received fromthe remote client device, the instructions comprising code for:

receiving notification of the event at the server;

determining an action, based on the event, to be performed by the remoteclient device on a graphical user interface (GUI) at the remote clientdevice or a remote view at the remote client device;

generating the message based on the determined action; and

directing the message to the remote client device,

wherein the GUI is a GUI for controlling an application running on theserver from the remote client device, and the remote view is a view ofthe application running on the server.

25. The machine-readable medium of clause 24, wherein the eventcomprises a click on a textbox, the action is to activate a graphicalkeyboard of the GUI at the remote client device, and the messagecomprises a command to activate the graphical keyboard.

26. The machine-readable medium of clause 24, wherein the eventcomprises a window opening at the server, the action is to display thewindow within the remote view at the remote client device, and themessage comprises coordinates and dimensions of the window.

27. The machine-readable medium of clause 24, wherein the eventcomprises a pointer on an object of the application at the server, theaction is to display substantially the entire object within the remoteview at the remote client device, and the message comprises coordinatesand dimensions of the object.

28. The machine-readable medium of clause 27, wherein the objectcomprises a picture, an icon, an animation, or video.

29. The machine-readable medium of clause 24, wherein the eventcomprises a pointer click on a video object, the action is to play avideo file associated with the video object at the remote client device,and the message comprises a uniform resource locator (URL) fordownloading the video file.

30. The machine-readable medium of clause 24, wherein the eventcomprises a dialog box opening at the server, the action is to displaythe dialog box within the remote view at the remote client device, andthe message comprises coordinates and dimensions of the dialog box.

31. The machine-readable medium of clause 30, wherein the instructionsfurther comprises code for:

receiving notification of when the dialog box closes at the server;

generating a second message informing the remote client device that thedialog box has closed; and

directing the second message to the remote client device.

32. The machine-readable medium of clause 24, wherein the instructionsfurther comprise code for:

receiving a command message from the remote client device; and

controlling the application on the server based on the received commandmessage.

33. The machine-readable medium of clause 32, wherein the instructionsfurther comprise code for:

resizing, moving or maximizing a view of the application on the serverbased on the received command message.

34. The machine-readable medium of clause 24, wherein the instructionsfurther comprise code for:

receiving command messages from the remote client devices;

directing the received command messages to one of a plurality ofdifferent applications on the server; and

controlling the one of the plurality of different applications on theserver based on received command messages.

35. An apparatus for providing, to a remote client device, a messagerelated to an event at the server, in which the event is in response toone or more user inputs received from the remote client device,comprising:

means for receiving notification of the event at the server (e.g.,2310-B of FIG. 23B);

means for determining an action, based on the event, to be performed bythe remote client device on a graphical user interface (GUI) at theremote client device or a remote view at the remote client device (e.g.,2320-B);

means for generating the message based on the determined action (e.g.,2330-B); and

means for directing the message to the remote client device (e.g.,2340-B),

wherein the GUI is a GUI for controlling an application running on theserver from the remote client device, and the remote view is a view ofthe application running on the server.

36. The apparatus of clause 35, wherein the event comprises a click on atextbox, the action is to activate a graphical keyboard of the GUI atthe remote client device, and the message comprises a command toactivate the graphical keyboard.

37. The apparatus of clause 35, wherein the event comprises a windowopening at the server, the action is to display the window within theremote view at the remote client device, and the message comprisescoordinates and dimensions of the window.

38. The apparatus of clause 35, wherein the event comprises a pointer onan object of the application at the server, the action is to displaysubstantially the entire object within the remote view at the remoteclient device, and the message comprises coordinates and dimensions ofthe object.

39. The apparatus of clause 38, wherein the object comprises a picture,an icon, an animation, or video.

40. The apparatus of clause 35, wherein the event comprises a pointerclick on a video object, the action is to play a video file associatedwith the video object at the remote client device, and the messagecomprises a uniform resource locator (URL) for downloading the videofile.

41. The apparatus of clause 35, wherein the event comprises a dialog boxopening at the server, the action is to display the dialog box withinthe remote view at the remote client device, and the message comprisescoordinates and dimensions of the dialog box.

42. The apparatus of clause 41, further comprising:

means for receiving notification of when the dialog box closes at theserver;

means for generating a second message informing the remote client devicethat the dialog box has closed; and

means for directing the second message to the remote client device.

43. The apparatus of clause 35, further comprising:

means for receiving a command message from the remote client device; and

means for controlling the application on the server based on thereceived command message.

44. The apparatus of clause 43, further comprising:

means for resizing, moving or maximizing a view of the application onthe server based on the received command message.

45. The apparatus of clause 35, further comprising:

means for receiving command messages from the remote client devices;

means for directing the received command messages to one of a pluralityof different applications on the server; and

means for controlling the one of the plurality of different applicationson the server based on the received command messages.

The subject technology is illustrated, for example, according to variousaspects described below. Numbered clauses are provided below forconvenience. These are provided as examples, and do not limit thesubject technology.

1. A system for communication and for rendering at a local client devicea composite view including a local graphical user interface (GUI) and aremote view associated with a remote application running on a remoteserver, comprising:

a remote view module configured to generate the remote view of theremote application based on display output data of the remoteapplication received at the local client device from the remote server;

a local view module configured to generate the local GUI for controllingthe remote application remotely from the local client device, configuredto control an area of the display output data of the remote applicationthat is displayed in the remote view, and configured to keep track ofcoordinates and dimensions of the area; and

a display module configured to provide, to a local display at the localclient device, a composite view comprising the local GUI and the remoteview of the remote application.

2. The system of clause 1, wherein the local view module is configuredto receive a message from the remote server including a command toactivate a graphical keyboard of the local GUI at the local clientdevice and configured to active the graphical keyboard in response tothe received message.

3. The system of clause 1, wherein the local view module is configuredto receive coordinates and dimensions of a window or dialog box at theremote server from the remote server, configured to compare thecoordinates and dimensions of the window or dialog box with thecoordinates and dimensions of the area of the display output datadisplayed in the remote view, and configured to adjust the area of thedisplay output data displayed in the remote view based on thecomparison.

4. The system of clause 3, wherein the local view module is configuredto adjust the area of the display output data displayed in the remoteview by scrolling the area.

5. The system of clause 3, wherein the local view module is configuredto adjust the area of the display output data displayed in the remoteview by zooming in or zooming out the area.

6. The system of clause 3, wherein the local view module is configuredto receive a second message from the remote server that the window ordialog box has been closed at the remote server and configured to undothe adjustment of the area of the display output data displayed in theremote view in response to the second message.

7. The system of clause 1, wherein the local view module is configuredto receive coordinates and dimensions of an object of the remoteapplication from the remote server, configured to compare thecoordinates and dimensions of the object with the coordinates anddimensions of the area of the display output data displayed in theremote view, and configured to adjust the area of the display outputdata displayed in the remote view based on the comparison.

8. The system of clause 7, wherein the local view module is configuredto adjust the area of the display output data displayed in the remoteview such that substantially the entire object is displayed in theremote view.

9. The system of clause 8, wherein the local view module is configuredto adjust the area of the display output data displayed in the remoteview by scrolling the area.

10. The system of clause 8, wherein the local view module is configuredto adjust the area of the display output displayed in the remote view byzooming in or zooming out the area.

11. The system of clause 1, wherein the local view module is configuredto receive a uniform resource locator (URL) from the remote server,configured to download a video file using the URL, and configured toplay the video file using a local media player application.

12. The system of clause 1, wherein the system is the local clientdevice comprising a processing system.

13. A method for rendering at a local client device a composite viewincluding a local graphical user interface (GUI) and a remote viewassociated with a remote application running on a remote server,comprising:

generating the remote view of the remote application based on displayoutput data of the remote application received at the local clientdevice from the remote server (e.g., 2410-A of FIG. 24A);

generating the local GUI for controlling the remote application remotelyfrom the local client device (e.g., 2420-A);

controlling an area of the display output data of the remote applicationthat is displayed in the remote view (e.g., 2430-A);

tracking coordinates and dimensions of the area (e.g., 2440-A); and

providing, to a local display at the local client device, a compositeview comprising the local GUI and the remote view of the remoteapplication (e.g., 2450-A).

14. The method of clause 13, further comprising:

receiving a message from the remote server including a command toactivate a graphical keyboard of the local GUI at the local clientdevice; and

activating the graphical keyboard in response to the received message.

15. The method of clause 13, further comprising:

receiving coordinates and dimensions of a window or dialog box at theremote server from the remote server;

comparing the coordinates and dimensions of the window or dialog boxwith the coordinates and dimensions of the area of the display outputdata displayed in the remote view; and

adjusting the area of the display output data displayed in the remoteview based on the comparison.

16. The method of clause 15, wherein adjusting the area of the displayoutput data displayed in the remote view comprises scrolling the area.

17. The method of clause 15, wherein adjusting the area of the displayoutput data displayed in the remote view comprises zooming in or zoomingout the area.

18. The method of clause 15, further comprising:

receiving a second message from the remote server that the window ordialog box has been closed at the remote server; and

undoing the adjustment of the area of the display output data displayedin the remote view in response to the second message.

19. The method of clause 13, further comprising:

receiving coordinates and dimensions of an object of the remoteapplication from the remote server;

comparing the coordinates and dimensions of the object with thecoordinates and dimensions of the area of the display output datadisplayed in the remote view; and

adjusting the area of the display output data displayed in the remoteview based on the comparison.

20. The method of clause 19, wherein the adjusting the area of thedisplay output data comprises adjusting the area of the display outputdata such that substantially the entire object is displayed in theremote view.

21. The method of clause 19, wherein the adjusting the area of thedisplay output data displayed in the remote view comprises scrolling thearea.

22. The method of clause 19, wherein the adjusting the area of thedisplay output data displayed in the remote view comprises zooming in orzooming out the area.

23. The method of clause 13, further comprising:

receiving a uniform resource locator (URL) from the remote server;

downloading a video file using the URL; and

playing the video file using a local media player application.

24. A machine-readable medium encoded with instructions for rendering ata local client device a composite view including a local graphical userinterface (GUI) and a remote view associated with a remote applicationrunning on a remote server, the instructions comprising code for:

generating the remote view of the remote application based on displayoutput data of the remote application received at the local clientdevice from the remote server;

generating the local GUI for controlling the remote application remotelyfrom the local client device;

controlling an area of the display output data of the remote applicationthat is displayed in the remote view;

tracking coordinates and dimensions of the area; and

providing, to a local display at the local client device, a compositeview comprising the local GUI and the remote view of the remoteapplication.

25. The machine-readable medium of clause 24, wherein the instructionsfurther comprise code for:

receiving a message from the remote server including a command toactivate a graphical keyboard of the local GUI at the local clientdevice; and

activating the graphical keyboard in response to the received message.

26. The machine-readable medium of clause 24, wherein the instructionsfurther comprise code for:

receiving coordinates and dimensions of a window or dialog box at theremote server from the remote server;

comparing the coordinates and dimensions of the window or dialog boxwith the coordinates and dimensions of the area of the display outputdata displayed in the remote view; and

adjusting the area of the display output data displayed in the remoteview based on the comparison.

27. The machine-readable medium of clause 26, wherein the adjusting thearea of the display output data displayed in the remote view comprisesscrolling the area.

28. The machine-readable medium of clause 26, wherein the adjusting thearea of the display output data displayed in the remote view compriseszooming in or zooming out the area.

29. The machine-readable medium of clause 26, wherein the instructionsfurther comprise code for:

receiving a second message from the remote server that the window ordialog box has been closed at the remote server; and

undoing the adjustment of the area of the display output data displayedin the remote view in response to the second message.

30. The machine-readable medium of clause 24, wherein the instructionsfurther comprise code for:

receiving coordinates and dimensions of an object of the remoteapplication from the remote server;

comparing the coordinates and dimensions of the object with thecoordinates and dimensions of the area of the display output datadisplayed in the remote view; and

adjusting the area of the display output data displayed in the remoteview based on the comparison.

31. The machine-readable medium of clause 30, wherein the adjusting thearea of the display output data comprises adjusting the area of thedisplay output data such that substantially the entire object isdisplayed in the remote view.

32. The machine-readable medium of clause 30, wherein the adjusting thearea of the display output data displayed in the remote view comprisesscrolling the area.

33. The machine-readable medium of clause 30, wherein the adjusting thearea of the display output displayed in the remote view compriseszooming in or zooming out the area.

34. The machine-readable medium of clause 24, wherein the instructionsfurther comprise code for:

receiving a uniform resource locator (URL) from the remote server;

downloading a video file using the URL; and

playing the video file using a local media player application.

35. An apparatus for rendering at a local client device a composite viewincluding a local graphical user interface (GUI) and a remote viewassociated with a remote application running on a remote server,comprising:

means for generating the remote view of the remote application based ondisplay output data of the remote application received at the localclient device from the remote server (e.g., 2410-B of FIG. 24B);

means for generating the local GUI for controlling the remoteapplication remotely from the local client device (e.g., 2420-B);

means for controlling an area of the display output data of the remoteapplication that is displayed in the remote view (e.g., 2430-B);

means for tracking coordinates and dimensions of the area (e.g.,2440-B); and

means for providing, to a local display at the local client device, acomposite view comprising the local GUI and the remote view of theremote application (e.g., 2450-B).

36. The apparatus of clause 35, further comprising:

means for receiving a message from the remote server including a commandto activate a graphical keyboard of the local GUI at the local clientdevice; and

means for activating the graphical keyboard in response to the receivedmessage.

37. The apparatus of clause 35, further comprising:

means for receiving coordinates and dimensions of a window or dialog boxat the remote server from the remote server;

means for comparing the coordinates and dimensions of the window ordialog box with the coordinates and dimensions of the area of thedisplay output data displayed in the remote view; and

means for adjusting the area of the display output data displayed in theremote view based on the comparison.

38. The apparatus of clause 37, wherein the means for adjusting the areaof the display output data displayed in the remote view comprises meansfor scrolling the area.

39. The apparatus of clause 37, wherein the means for adjusting the areaof the display output data displayed in the remote view comprises meansfor zooming in or zooming out the area.

40. The apparatus of clause 37, further comprising:

means for receiving a second message from the remote server that thewindow or dialog box has been closed at the remote server; and

means for undoing the adjustment of the area of the display output datadisplayed in the remote view in response to the second message.

41. The apparatus of clause 35, further comprising:

means for receiving coordinates and dimensions of an object of theremote application from the remote server;

means for comparing the coordinates and dimensions of the object withthe coordinates and dimensions of the area of the display output datadisplayed in the remote view; and

means for adjusting the area of the display output data displayed in theremote view based on the comparison.

42. The apparatus of clause 41, wherein the means for adjusting the areaof the display output data comprises means for adjusting the area of thedisplay output data such that substantially the entire object isdisplayed in the remote view.

43. The apparatus of clause 41, wherein the means for adjusting the areaof the display output data displayed in the remote view comprises meansfor scrolling the area.

44. The apparatus of clause 41, wherein the means for adjusting the areaof the display output displayed in the remote view comprises means forzooming in or zooming out the area.

45. The apparatus of clause 35, further comprising:

means for receiving a uniform resource locator (URL) from the remoteserver;

means for downloading a video file using the URL; and

means for playing the video file using a local media player application.

Portability of a remote desktop session may be desirable to a user forseveral reasons. For example, a user may want to forward a live remotesession from a phone to her computer or vice versa, or a user may wantto use a display device, such as a projector, display a live sessionfrom a small-screen device such as a mobile phone. While some techniques(e.g., the remote desktop protocol or RDP) facilitate portability bypreserving a remote desktop session and allowing a user to log out fromone client device and log back into the preserved remote desktop sessionfrom another client device, these techniques fail to provide acomprehensive solution for authentication and portability of remotedesktop sessions. For example, the session hand-off is cumbersome andtime-consuming because it requires a user to submit to another biometricscan and/or go through the additional operations of logging into a newsession.

In one aspect of the disclosure, a configuration of the subjecttechnology enables the secure passing of user credentials and remotedesktop sessions between a client device (e.g., a smartphone such as aBlackberry or iPhone) and a host device (e.g., a thin client or aworkstation).

FIG. 25 illustrates a simplified diagram of a system 2500 in accordancewith one configuration of the present disclosure. The system 2500 mayinclude one or more client devices 2502 in communication with a remoteserver 2504 over a first communication network 2510. The remote server2504 is also communicatively coupled with a host device 2506 via asecond communication network 2512. The host device 2506 and the clientdevice 2502 may be able to communicatively couple over a thirdcommunication network 2508.

By way of illustration and not limitation, a client device 2502, a hostdevice 2506 and a remote server 2504 can represent a computer, a mobilephone, a laptop computer, a thin client device, a personal digitalassistant (PDA), a portable computing device, or a suitable device witha processor. In one example, a client device 2502 is a smartphone (e.g.,iPhone, Android phone, Blackberry, etc.), a host device 2506 is a thinclient device and a remote server 2504 is a server computer. In certainconfigurations, a client device 2502, a remote server 2504 and a hostdevice 2506 can each represent an audio player, a game console, acamera, a camcorder, an audio device, a video device, a multimediadevice, or a device capable of supporting a connection to a remoteserver. In one example, a client device 2502, a remote server 2504 and ahost device 2506 can be mobile. In another example, a client device2502, a remote server 2504 and a host device 2506 can be stationary.

By way of illustration and not limitation, each of communicationnetworks 2510 and 2512 may be wired or wireless communication networksand may have any geographic reach (e.g., a personal area network, alocal area network, or a wide area network). The networks 2510 and/or2512 may use the internet protocol (IP) packet format or any otherformat for data communication. The networks 2510 and/or 2512 maycomprise one or more heterogeneous networks communicating on differentphysical media, and may be managed by one or more network operators. Inone example, communication networks 2510 and/or 2512 may be a wirelesswide area network comprising a cellular telephony network such as a codedivision multiple access (CDMA) or a Global System for MobileCommunication (GSM) network. In one example, communication networks 2510and/or 2512 may be the Internet.

By way of illustration and not limitation, a communication network 2508may be a wired or wireless short range communication network such as oneof the family of Institute of Electrical and Electronics Engineers(IEEE) 802.x communication networks, a Bluetooth network, a universalserial bus (USB) connection, a near-field communication (NFC) network, aFireWire connection, or another type of connection or network. In oneexample, communication network 2508 may be a connection or networkwithin a distance of 1 foot, 5 feet, 10 feet, 15 feet, 20 feet, 50 feetor 100 feet (e.g., devices 2502 and 2506 are located within 1 foot, 5feet, 10 feet, 15 feet, 20 feet, 50 feet or 100 feet of each other). Incertain configurations, network 2508 may be of a type as described abovefor a communication network 2510 or 2512.

In one aspect, the term “network” may, for example, refer to aconnection or vice versa. In one aspect, the term “a network” may, forexample, refer to one or more networks, and the term “a connection” may,for example, refer to one or more connections.

FIG. 26A illustrates a simplified block diagram of an embodiment 2601 ofa client device 2502 in accordance with certain embodiments of thepresent disclosure. The operating system (OS) 2620 is in communicationwith a client trusted relationship module 2602, a client remote sessionlogin information module 2604 and a client session launch module 2606.Features and functions of these modules according to certain embodimentsof the present disclosure may be implemented in software, in hardwareand/or in combination and are further described in the disclosure.

FIG. 26B illustrates a simplified block diagram of a configuration 2603of a client device 2502 in accordance with certain embodiments of thepresent disclosure. The operating system (OS) 2620 is in communicationwith a client trusted relationship module 2602, a client remote sessionlogin information module 2604, a client session launch module 2606 and aclient remote session termination module 2608. Features and functions ofthese modules according to certain embodiments of the present disclosuremay be implemented in software, in hardware and/or in combination andare further described in the disclosure.

FIG. 26C illustrates a simplified block diagram of a configuration 2605of a client device 2502 in accordance with certain embodiments of thepresent disclosure. The operating system (OS) 2620 is in communicationwith a client trusted relationship module 2602, a client forwardingmodule 2610 and a client session launch module 2606. Features andfunctions of these modules according to certain aspects of the presentdisclosure may be implemented in software, in hardware and/or incombination and are further described in the disclosure.

FIG. 26D illustrates a simplified block diagram of an embodiment 2607 ofa client device 2502 in accordance with certain embodiments of thepresent disclosure. The operating system (OS) 2620 is in communicationwith a client trusted relationship module 2602, a client logincredentials module 3212 and a client session launch module 2606.Features and functions of these modules according to certain embodimentsof the present disclosure may be implemented in software, in hardwareand/or in combination and are further described in the disclosure.

FIG. 26E illustrates a simplified block diagram of an embodiment of aclient trusted relationship module 2602 in accordance with certainembodiments of the present disclosure. The illustrated configurationcomprises a client trusted relationship module 2602, a client logincredentials module 3212 and a client session launch module 2606.Features and functions of these modules according to certain embodimentsof the present disclosure may be implemented in software, in hardwareand/or in combination and are further described in the disclosure.

FIG. 27A illustrates a simplified block diagram of an embodiment 2701 ofa host device 2506 in accordance with certain embodiments of the presentdisclosure. The operating system (OS) 2720 is in communication with ahost trusted relationship module 2702, a host login information module2704 and a host remote session module 2706. Features and functions ofthese modules according to certain embodiments of the present disclosuremay be implemented in software, in hardware and/or in combination andare further described in the disclosure.

FIG. 27B illustrates a simplified block diagram of an embodiment 2703 ofa host device 2506 in accordance with certain embodiments of the presentdisclosure. The operating system (OS) 2720 is in communication with ahost trusted relationship module 2702, a host display data module 2710and a host remote session view module 2706. Features and functions ofthese modules according to certain embodiments of the present disclosuremay be implemented in software, in hardware and/or in combination andare further described in the disclosure.

FIG. 27C illustrates a simplified block diagram of an embodiment 3305 ofa host device 2506 in accordance with certain embodiments of the presentdisclosure. The operating system (OS) 2720 is in communication with ahost trusted relationship module 2702, a host login information module2704, a host remote session module 2706, and a host remote sessiontermination module 2712. Features and functions of these modulesaccording to certain embodiments of the present disclosure may beimplemented in software, in hardware and/or in combination and arefurther described in the disclosure.

FIG. 28 shows an exemplary process implemented at a client device 2502.At operation 2800, the client trusted relationship module 2602 operatingat a client device 2502 facilitates establishing a trusted relationshipwith a host device 2506. Upon establishment of a trusted relationshipwith a host device 2506, the client remote session login informationmodule then, in operation 2802, provides remote session logininformation to the host device 2506. The remote session logininformation comprises information that would be used by the clientdevice 2502 to establish a remote session with a remote server 2504. Inone aspect, the remote session login information may contain an identityof the remote server 2504 such as a universal resource locater (URL)address or an internet protocol (IP) address. In another aspect, theremote session login information may comprise an identity of the useroperating the client device 2502 and confidential informationauthenticating the user. The confidential information may be, forexample, in the form of a password or a biometric signature of the user.The remote session login information may be encrypted or in the clear.When encrypted, the remote session login information may be encryptedsuch that the remote server 2504 may be able to decrypt and use it toestablish a remote session, but the host device 2506 may not be able todecrypt the remote session login information. This may be achieved by,for example, encrypting the remote session login information using apublic key of the remote server 2506 such that only the remote server2506 that is in possession of the corresponding private key can decrypta message encrypted using the public key. The remote session logininformation may also contain an identification of a remote session. Suchidentification may be used by the remote server 2504 to restore thestate of a previously established remote session between the clientdevice 2502. The remote session login information may also includeinformation such as a client's custom session information such as size,color and font preference for the remote session graphical userinterface (GUI).

Still referring to FIG. 28, in certain embodiments, the trustedrelationship established at operation 2800 may be responsive to the typeof physical layer connectivity between the client device 2502 with thehost device 2506. For example, if the client device 2502 and the hostdevice 2506 are coupled by a wired connection, such as a universalserial bus (USB) or a FireWire (IEEE-1394) connection, then the trustedrelationship module 2602 may consider that a trusted relationship existsbetween the client device 2502 and the host device 2506. A practicalreason for considering this to be a trusted relationship may be thatbecause both USB and FireWire generally require a user to physicallyplug in the connection wire, the wired connection may indicate that auser is controlling both the client device 2502 and the host device2506, thereby establishing the trusted relationship.

In certain configurations, the client trusted relationship module 2602may, in addition to or in lieu of being responsive to the type ofphysical layer connectivity between the client device 2502 and the hostdevice 2506, perform message exchanges with the host device 2506 toestablish a trusted relationship with the host device 2506. By way ofexamples, such a need may arise when the communication between theclient device 2502 and the host device 2506 may be accessible to otherdevices. The need to perform these message exchanges may arise forexample, but not limited to, when the client device 2502 and the hostdevice 2506 communicate via a wireless network (e.g., Bluetooth or802.11a/b/c/g family of standards) or communicate on a shared wire(e.g., the Ethernet).

Accordingly, in certain configurations, the trusted relationship module2602 may perform a digital handshake (e.g., paired message exchangessuch as request/response or message/acknowledgement) with the hostdevice 2506. For example, the client trusted relationship module 2602may be configured to respond to a query message from the host device2506. This query/response mechanism may be used by the client trustedrelationship module 2602 to maintain the trusted relationship betweenthe host device 2506 and the client device 2502. In one configuration,as illustrated in FIG. 26E, the client trusted relationship module 2602may comprise a client presence sensing module 2622, a client pairingmodule 2624 and a client link security module 2626. The client presencesensing module 2622 may sense the presence of a host device 2506. Incertain configurations, the client presence sensing module 2622 mayachieve this by detecting physical connection of a cable at aninput/output port of the client device 2502. The client presence sensingmodule 2622 may use any of several well-known techniques such asdetecting impedance mismatch or signal detection. In certainconfigurations, the client presence sensing module 2622 may detectpresence of a host device 2506 by transmitting a query message on thecommunication medium and listening for a response. In certainconfigurations, the client presence sensing module 2622 may detectpresence of a host device 2506 by listening for a message transmissionfrom the host device 2506.

In one aspect, the client presence sensing module 2622 may furtherestimate proximity of a host device 2506 to the client device 2502. Incertain configurations, the client presence sensing module 2622 maydetermine the proximity responsive to a calculated a signal strengthparameter. Accordingly, the client presence sensing module 2622 mayestimate received signal strength of a transmission received from thehost device 2506. In one aspect, the client presence sensing module 2622may compare the calculated signal strength against a threshold value tomake a determination of whether the hose device 2506 is in proximity ofthe client device 2502 or not. In certain configurations, the clientpresence sensing module 2622 may consider that a host device 2506 is inthe proximity of the client device 2502 only if the received signalpower from a host device 2506 is above the threshold value. In certainconfigurations, the client presence sensing module 2622 may compare thetime difference between when a signal was sent and when a correspondingresponse was received to calculate a round trip delay time forcommunication with the host device 2506. In one aspect, the clientpresence sensing module may consider that a host device 2506 is in theproximity of the client device 2502 only if the round trip delay time isbelow a time threshold value. The

The client pairing module 2624 facilitates performing message exchangesto pair of a client device 2502 with a host device 2506. In certainconfiguration, the client pairing module 2624 may check if the clientdevice 2502 and the host device 2506 were connected to each other via awired connection such as a USB connection and may deem the client device2502 and the host device 2506 as being paired to each other. In certainconfigurations, the client pairing module 2624 may participate in apairing operation involving exchange of messages facilitating encryptedcommunication between the client device 2502 and the host device 2506.As an example, when the short range network 2508 between the clientdevice 2502 and the host device 2506 comprises a Bluetooth connection,the client pairing module 2624 may participate in a pairing operationwherein the client pairing module receives a challenge message from thehost device 2506. The client pairing module 2624 decrypts the receivedchallenge message using a locally stored private key and facilitatespresenting the result of the decryption to a user on a display attachedto the client device 2502. The user may then read the result and enterit on the host device 2506 to achieve pairing.

The client link security module 2626 may perform exchange of encryptioninformation with the host device 2506 to further achieve securecommunication. The exchange may include presenting a public key ordigital certificate to the host device 2506, receiving, and validating apublic key or a digital certificate from the host device 2506. After theexchange of encryption information, the client link security module 2626may achieve link security by encrypting subsequent communication withthe host device 2606. The client link security module 2626 may use avariety of well-known data encryption techniques, including, but notlimited to, encryption using digital key based encryption and hashing.

FIG. 29 shows an exemplary process implemented at a host device 2506. Inoperation 2900, a host trusted relationship module 2702 establishes atrusted relationship with a client device 2502. In operation 2902, ahost login information module 2704 obtains remote session logininformation from a client device 2502. In operation 2904, a host sessionlaunch module 2706 launches a remote session with a remote server 2504,using the remote session login information received from the clientdevice 2502. In certain configurations, the host session launch module2706 may launch a remote session automatically (i.e., without any userinput). In certain configurations, the host session launch module 2706may prompt a user to confirm launching a remote session.

Still referring to FIG. 29, in certain embodiments, the host trustedrelationship established at operation 2900 may be responsive to the typeof physical layer connectivity between the client device 2502 with thehost device 2506. For example, if the client device 2502 and the hostdevice 2506 are coupled by a wired connection, such as a universalserial bus (USB) or a FireWire (IEEE-1394) connection, then the hosttrusted relationship module 2702 may consider that a trustedrelationship exists between the client device 2502 and the host device2506. A practical reason for considering this to be a trustedrelationship may be that because both USB and FireWire generally requirea user to physically plug in the connection wire, the wired connectionmay indicate that a user is controlling both the client device 2502 andthe host device 2506, thereby establishing the trust.

In certain configurations, the host trusted relationship module 2702may, in addition to or in lieu of being responsive to the type ofphysical layer connectivity between the client device 2502 and the hostdevice 2506, perform message exchanges with the client device 2502 toestablish a trusted relationship with the client device 2502. By way ofexample, such a need may arise when the communication between the clientdevice 2502 and the host device 2506 may be accessible to other devices.The need to perform these message exchanges may arise for example, butnot limited to, when the client device 2502 and the host device 2506communicate via a wireless network (e.g., Bluetooth or 802.11a/b/c/gfamily of standards) or communicate on a shared medium (e.g., theEthernet) where other devices may be able to sense the signalscommunicating messages between the client device 2502 and the hostdevice 2506.

Accordingly, in certain configurations, the host trusted relationshipmodule 2702 may perform a digital handshake (e.g., paired messageexchanges such as request/response or message/acknowledgement) with theclient device 2502. In one configuration, as illustrated in FIG. 27C,the host trusted relationship module 2702 may comprise a host presencesensing module 2722, a host pairing module 2724 and a host link securitymodule 2726. The host presence sensing module 2722 may sense thepresence of a client device 2502. In certain configurations, the hostpresence sensing module 2722 may achieve this by detecting physicalconnection of a cable at an input/output port of the host device 2506.The host presence sensing module 2722 may use any of several well-knowntechniques such as detecting impedance mismatch or signal detection. Incertain configurations, the host presence sensing module 2722 may detectthe presence of a client device 2502 by transmitting a query message onthe communication medium and listening for a response. In certainconfigurations, the host presence sensing module 2722 may detectpresence of a client device 2502 by listening for a message transmissionfrom the client device 2502.

In one aspect, the host presence sensing module 2722 may furtherestimate proximity of a client device 2502 to the host device 2506. Incertain configurations, the host presence sensing module 2722 maydetermine the proximity responsive to a calculated signal strengthparameter based on the signal power received from the client device2502. In one aspect, the host presence sensing module 2722 may comparethe calculated signal strength against a threshold value to make adetermination of whether the client device 2502 is in proximity of thehost device 2506. In certain configurations, the host presence sensingmodule 2722 may consider that a client device 2502 is in the proximityof the host device 2506 only if the received signal power from theclient device 2502 is above the threshold value. In certainconfigurations, the host presence sensing module 2722 may compare thetime difference between when a signal was sent and when a correspondingresponse was received to calculate a round trip delay time forcommunication with the client device 2502. In one aspect, the hostpresence sensing module may consider that a client device 2502 is in theproximity of the host device 2506 only if the round trip delay time isbelow a time threshold value.

The host pairing module 2724 facilitates pairing of a client device 2502with a host device 2506. In certain configuration, the host pairingmodule 2724 may check if the client device 2502 and the host device 2506were connected to each other via a wired connection such as a USBconnection and may consider the client device 2502 and the host device2506 as being paired to each other if they were connected via a wiredconnection. In certain configurations, the host pairing module 2724 mayparticipate in a pairing operation involving exchange of messagesfacilitating encrypted communication between the client device 2502 andthe host device 2506. As an example, when the short range network 2508between the client device 2502 and the host device 2506 comprises aBluetooth connection, the host pairing module 2724 may participate in apairing operation wherein the host pairing module 2724 generates achallenge message and sends the challenge message to a client device2502. The client device 2502 may display a decrypted version of thechallenge message to a user on a display attached to the client device2502. The user may then read the result and enter the result on the hostdevice 2506. When a user enters the result, the host pairing module 2724then verifies if the entered result matches the seed used to generatethe challenge message, and if there is a match, the host pairing module2724 may then indicate that the host and client devices 2506, 2502 werepaired. Once the host device 2506 and the client device 2502 are thuspaired, the host device then may allow further communication with theclient device.

The host link security module 2726 may perform exchange encryptioninformation with the client device 2502 to further achieve a securecommunication link by encrypting communication to the client device2602. The host link security module 2726 may use a variety of well-knowndata encryption techniques, including, but not limited to, encryptionusing digital key based encryption and hashing.

FIG. 30 shows an exemplary process implemented at a host device 2506.After a host session launch module 2706 establishes a remote sessionwith a remote server 2504, using the remote session login informationreceived from a client device 2502, during the operation 3000 of theremote session, the host trusted relationship module may 2702 monitor,at operation 3002, if the trusted relationship still exits with theclient device 2502 from whom the remote session login information wasreceived. The host trusted relationship module 2502 allows the remotesession to continue if a trusted relationship exits (“YES” branch ofoperation 3002). If the host trusted relationship module 2702 determinesthat trusted relationship with the client device 2502 does not exist(“NO” branch of operation 3002), then the host trusted relationshipmodule 2702 may signal the host remote session termination module 2712to facilitate termination of the remote session with the remote server2504.

FIG. 31 shows an exemplary process implemented at a client device 2502.In operation 3102, a client session launch module 2606 establishes aremote session with a remote server 2504. During the life time of theremote session, the client trusted relationship module 2602 mayestablish, at operation 3104, a trusted relationship with a host device2506. After a trusted relationship is thus established, the clientremote session login information module 2604, at operation 3106,provides remote session login information to the host device 2506. Atoperation 3108, the client session termination module terminates theremote session between the client device 2502 and the remote server2504. In certain configurations, the client session termination module2608 may perform operation 3108 without receiving any indication fromthe host device 2506 regarding whether the host device 2506 launched aremote session with the remote server, based on the remote session logininformation received. In another configurations, the client sessiontermination module 2608 may perform operation 3108 after receiving anindication that the host device 2506 successfully started a remotesession with the remote server 2504 based on the remote session logininformation received from the client device 2502.

FIG. 32 shows an exemplary process implemented at a host device 2506. Inoperation 3202, the host trusted relationship module 2702 may establisha trusted relationship with a client device 2504. The host device 2506may perform operation 3202 substantially identically to operation 2900described above. In operation 3204, the host login information module2704 receives remote session login information from the client device2504. The remote session login information may be substantiallyidentical to the description provided above with relation to operation2902. In operation 3206, the host session launch module 2706 may launcha remote session with the remote server 2504 using the received remotesession login information. The host session launch module 2706 mayimplement operation 3806 substantially identical to operation 2904described above.

FIG. 33 shows an exemplary process implemented at a client device 2502.In operation 3302, a client session launch module 2606 establishes aremote session with a remote server 2504. The client session launchmodule 2606 may perform operation 3302 substantially identical tooperation 3102 described above. During the life time of the remotesession, the client trusted relationship module 2602 may establish, atoperation 3904, a trusted relationship with a host device 2506. Theclient trusted relationship module 2602 may implement operation 3304substantially identical to operation 2800 described above. After atrusted relationship is thus established, the client forwarding module2610, at operation 3306, selectively provides session display data forthe session with the remote server 2504 to the host device 2506.

In one configuration, the client device 2502 may be configured todisplay a composite view of the remote session on the client device2502. The composite view may include (a) a local graphical userinformation (GUI), displaying, for example, toolbars and headers, and(b) a remote application view, presenting, for example, informationreceived from the remote server 2504 in the form of display data. In oneconfiguration, the client forwarding module 2610 may filter out datarelated to local GUI and selectively forward the display data related tothe remote session to the host device 2506 and display the local GUI onthe client device 2502. In such configurations, a user may be able tocontrol a remote session by interacting with local GUI controls from theclient device 2502, and be able to display the remote application viewon the host device's display. In another configuration, the clientforwarding module 2610 may forward all information received from theremote server 2504, thereby providing a host computer 2506 withsufficient information to be able to display both the local GUIinformation and remote application view. In other configurations, theclient forwarding module 2610 may be configured to send a subset of thereceived remote session information (e.g., only audio or only videoinformation).

FIG. 34 shows an exemplary process implemented at a host device 2506. Inoperation 3402, the host trusted relationship module 2702 may establisha trusted relationship with a client device 2504. The host device 2506may perform operation 3402 substantially identically to operation 2900described above. In operation 4004, the host display data module 2710receives display data information related to a remote session from theclient device 2504. The display data information may comprise protocolcommands for displaying on a monitor. For example, the display datainformation may include data in a remote desktop protocol (RDP) format.In operation 3406, the host remote session view module 2706 displays, inthe form of a remote application view, the received display data on alocal monitor. In certain configurations, the remote application viewmay include GUI control objects (e.g., toolbar and other menu items) forthe application being run remotely at the remote server 2504. In suchconfigurations, the host remote session view module 2706 may communicateany GUI events by a user of the host device 2506. For example, in oneaspect, if a user scrolls the application GUI being displayed at thehost device 2506, the host remote session view module may convey thecorresponding control information to the client device 2502. In certainconfigurations, while the host device 2506 is able to display remoteapplication view for an application running on a remote server 2504, thehost device 2506 may not be directly able to communicate with the remoteserver 2504 regarding the remote session because the host device 2506may not have information regarding identity of the remote server 2504hosting the remote application. Therefore, in one aspect, the hostdevice 2506 acts as a display device for the remotely runningapplication, while the client device 2502 is used to remotely controlthe application running on the remote server 2504.

FIG. 35 shows an exemplary process implemented at a client device 2502.In operation 3502, the client trusted relationship module 2602establishes a trusted relationship with a host device 2506. The clienttrusted relationship module 2602 may implement operation 4102substantially identical to the operation 2800 described above. After atrusted relationship is thus established, the client login informationreceiver module 2712 may receive, in operation 3504, remote sessionlogin information from the host device 2506. The remote session logininformation may be substantially as described with respect to operation2802 above. The client session launch module 2606 may use the receivedremote session login information to facilitate launching a remotesession, in operation 3506, with the remote server 2504 identified inthe remote session login information. In certain configurations, theremote server 2504 may be configured to save a state of the remotesession, and the user experience for operation 3506 may be that the useris continuing a previous remote session with the remote server 2504 fromthe client device 2502.

FIG. 36 shows an exemplary process implemented at a host device 2506. Inoperation 3602, the host session launch module 2706 establishes a remotesession with a remote server 2504. During the life time of the remotesession, in operation 3604, the host trusted relationship module 2702may establish a trusted relationship with a client device 2504. The hostdevice 2506 may perform operation 3604 substantially identically tooperation 2900 described above. In operation 3606, the host logininformation module 2704 may provide remote session login information forthe remote session to the client device 2502. The remote session logininformation may be substantially as described with respect to operation3204. In operation 3608, the host remote session termination module maythen terminate the remote session with the remote server 2504. Incertain configuration, the host remote session termination module mayterminate the remote session after receiving an indication from theclient device 2502 that the remote session was continued from the clientdevice 2502 (e.g., as in operation 3606 described above). In otherconfigurations, the termination operation 3608 may occur without anindication from the client device 2502.

Referring to FIG. 37, exemplary aspects of the present disclosure arenow provided below for a system 3700 wherein the client device 2502 is aBluetooth-capable mobile phone, the remote server 2504 is an applicationserver (e.g., a web server or a corporate application server), and thehost device 2506 is a thin client device. The first network 2510 is adata network comprising cellular telephone network. The second network2512 is the Internet. The short range network 3708 comprises theBluetooth protocol. It will be understood by those skilled in the artthat while the messages exchanged among entities in system 3700 arerepresented as single arrows between the entities, the arrows mayrepresent one or more actual signal transmissions and there may be othermessages communicated between the entities performing other tasks duringthis time.

Example of Use Case A

FIG. 38 illustrates exemplary messages exchanged for a use case in whicha host device 2506 establishes a remote session with a remote server2504 using login information received from a client device 2502.

A message 3802 is from the client device 2502 to the host device 2506via the network 3708 to authentication the user via a Bluetoothconnection:

(i) When the user carrying the client device 2502 is nearby the hostdevice 2506, the two devices 2502, 2504 are paired (e.g., as describedabove in with respect to operation 2900). In message 3802, the clientdevice 2502 identifies the user operating the client device 2502 to thehost device 2506.

(ii) The host device 2506 then finds the user name login of the useridentified by the client device 2502 in message 3802 and presents alogin prompt to the user on the host device's display.

(iii) After the user successfully logs in the host device 2506, the hostdevice 2506 then sends, in a security request message 3804, a code tothe client device 2502, which can only be decrypted via the private keythat is installed on the client device 2502.

(iv) The client device 2502 then displays the code on its screen and theuser enters the code in the host device 2506 (security response 3806)and authentication occurs and the user logs in. The host device 2506establishes a remote session 3808 with the remote server 2504.

(v) The user has the option to terminate the remote session anytime fromthe client device 2502 (e.g., by ending the trusted relationship, asdescribed in operation 3002) or the host device 2506 directly (e.g., byending the remote session via GUI).

(vi) If the user walks away, then the remote session automatically isdisconnected due to detection of lack of the presence of the clientdevice 2502 (e.g., loss of proximity, as described with respect tooperation 3002).

Example of Use Case B

FIG. 39 illustrates exemplary messages exchanged in which a user is ableto forward a live remote session from the client device 2502 to the hostdevice 2506 (passing on credentials) via the connection:

(i) Message 3902 represents that the user has already started a remotesession between the client device 2502 and the remote server 2504. Inthis example, the remote server 2504 executes (or runs) one or moreapplications during the remote session, and the client device 2502 doesnot execute the one or more applications.

(ii) When the user is close to the host device 2506, the client device2502 identifies the user and forwards login information to the hostdevice 2504, in message 3904, without any further interaction with thecomputer.

(iii) Using the login information received in message 3904, the hostdevice 2506 creates a new remote session (message 3906) with the remoteserver 2504.

Example of Use Case C

FIG. 40 illustrates exemplary messages exchanged when a user forwards alive remote session from the client device 2502 to the host device 2506by passing on display data via the Bluetooth connection.

(i) Message 4002 indicates that the user has already started a remotesession between the client device 2502 and the remote server 2504 whereapplications are executed.

(ii) When the user is close to the host device 2506, the client device2502 identifies the user and forwards login information to the hostdevice 2504, in message 4004, without any further interaction with thecomputer.

(iii) The host device 2506 starts decoding the received display data(e.g., protocol commands when the remote session is implemented usingthe remote desktop protocol), indicated as message 4006, and startsdisplaying the results on the host device's display.

(iv) It will be appreciated by those skilled in the art that, in theillustrated example, the client device 2502 is the broker in between thehost device 2506 and the remote server 2504 and essentially never passeson the user login information to the host device 2506 and functions asthe gateway between the remote server 2504 and the host device 2506.

Example of Use Case D

FIG. 41 illustrates exemplary messages exchanged when a user forwards alive remote session from the host device 2506 to the client device 2502by passing login information via the Bluetooth, connections so the usercan receive the “remote session-to-go”:

(i) Message 4102 indicates that the user has already established aremote session between the remote server 2504 and the host device 2506.

(ii) Message 4104 indicates that the client device 2502 is paired withthe host device 2506 via a Bluetooth connection

(iii) Message 4106 indicates that the host device 2506 passes on theuser login information to the client device 2502.

(iv) Message 4108 indicates that the client device 2502 re-establishes aremote session with the remote server 2504, using the login informationreceived in message 4106, after the user disconnects or walks away fromthe host device 2506 and the live remote session runs on the clientdevice 2502 and no additional authentication is needed from the user onthe client device 2502.

The subject technology is illustrated, for example, according to variousaspects described below. Numbered clauses are provided below forconvenience. These are provided as examples, and do not limit thesubject technology.

1. A system at a client device side for communication and forfacilitating establishing a remote session between a host device and aremote server, the system comprising:

a client trusted relationship module configured to facilitateestablishing a trusted relationship between a client device and the hostdevice;

a client remote session login information module configured to provideremote session login information to the host device to enable the hostdevice to establish a first remote session with the remote server; and

a client session launch module configured to launch a second remotesession with the remote server using the login information.

2. The system of clause 1, wherein the client trusted relationshipmodule is configured to respond to a query message from the host device.

3. The system of clause 1, wherein the client device is configured tocommunicate with the host device over a wireless connection.

4. The system of clause 1, wherein the client trusted relationshipmodule comprises:

a client presence sensing module configured to sense a presence of theclient device near the host device;

a client pairing module configured to facilitate performing messageexchanges to pair the client device with the host device; and

a client link security module configured to perform exchange ofencryption information with the host device.

5. The system of clause 4, wherein the client presence sensing module isconfigured to estimate a received signal strength of a transmissionreceived from the host device.

6. The system of clause 4, wherein the client presence sensing module isconfigured to calculate a round trip delay time for communication withthe host device.

7. The system of clause 1, wherein the system is the client devicecomprising a processing system and a display.

8. A method at a client device side for communication and forfacilitating establishing a remote session between a host device and aremote server, the method comprising:

facilitating establishing a trusted relationship between the clientdevice and the host device (e.g., 4200-A of FIG. 42);

providing remote session login information to the host device to enablethe host device to establish a first remote session with the remoteserver (e.g., 4202-A of FIG. 42); and

launching a second remote session with the remote server using the logininformation (e.g., 4204-A of FIG. 42).

9. The method of clause 8, wherein the facilitating establishing thetrusted relationship comprises responding to a query message from thehost device.

10. The method of clause 8, wherein the client device communicates withthe host device over a wireless connection.

11. The method of clause 8, wherein the facilitating establishing thetrusted relationship comprises:

sensing a presence of the client device near the host device;

performing message exchanges to pair the client device with the hostdevice; and

performing exchange of encryption information with the host device.

12. The method of clause 11, wherein the sensing the presence comprisesestimating a received signal strength of a transmission received fromthe host device.

13. The method of clause 11, wherein the sensing the presence comprisescalculating a round trip delay time for communication with the hostdevice.

14. A machine-readable medium encoded with instructions for execution ata client device side for communication and for facilitating establishinga remote session between a host device and a remote server, theinstructions comprising code for:

facilitating establishing a trusted relationship between the clientdevice and the host device;

providing remote session login information to the host device to enablethe host device to establish a first remote session with the remoteserver; and

launching a second remote session with the remote server using the logininformation.

15. The machine-readable medium of clause 8, wherein the code forfacilitating establishing the trusted relationship comprises code forresponding to a query message from the host device.

16. The machine-readable medium of clause 8, wherein the client deviceis configured to communicate with the host device over a wirelessconnection.

17. The machine-readable medium of clause 8, wherein the code forfacilitating establishing the trusted relationship comprises code for:

sensing a presence of the client device near the host device;

performing message exchanges to pair the client device with the hostdevice; and

performing exchange of encryption information with the host device.

18. The machine-readable medium of clause 17, wherein the code forsensing the presence comprises code for estimating a received signalstrength of a transmission received from the host device.

19. The machine-readable medium of clause 17, wherein the code forsensing the presence comprises code for calculating a round trip delaytime for communication with the host device.

20. An apparatus for communication and for facilitating establishing aremote session between a host device and a remote server, the apparatuscomprising:

means for facilitating establishing a trusted relationship between theclient device and the host device (e.g., 4200-B of FIG. 44);

means for providing remote session login information to the host deviceto enable the host device to establish a first remote session with theremote server (e.g., 4202-B of FIG. 44); and

means for launching a second remote session with the remote server usingthe login information (e.g., 4204-B of FIG. 44).

21. The apparatus of clause 20, wherein the means for facilitatingestablishing the trusted relationship comprises means for responding toa query message from the host device.

22. The apparatus of clause 20, configured to communicate with the hostdevice over a wireless connection.

23. The apparatus of clause 20, wherein the means for facilitatingestablishing the trusted relationship comprises:

means for sensing a presence of the client device near the host device;

means for performing message exchanges to pair the client device withthe host device; and

means for performing exchange of encryption information with the hostdevice.

24. The apparatus of clause 23, wherein the means for sensing thepresence comprises means for estimating a received signal strength of atransmission received from the host device.

25. The apparatus of clause 23, wherein the means for sensing thepresence comprises means for calculating a round trip delay time forcommunication with the host device.

26. A system at a host device side for communication and forfacilitating establishing a remote session between a host device and aremote server, comprising:

a host trusted relationship module configured to facilitate establishinga trusted relationship between a client device and the host device;

a host login information module configured to obtain remote sessionlogin information from the client device; and

a host remote session module configured to facilitate establishing aremote session, using the remote session login information, with theremote server,

wherein the host device is configured to communicate with the clientdevice via a wireless connection.

27. The system of clause 26, wherein the host trusted relationshipmodule comprises:

a host presence sensing module configured to sense a presence of theclient device near the host device;

a host pairing module configured to pair the client device with the hostdevice; and

a host link security module configured to perform exchange of encryptioninformation with the host device.

28. The system of clause 26, wherein the host trusted relationshipmodule is configured to monitor if a trusted relationship exists withthe client device.

29. The system of clause 26, further comprising:

a host remote session termination module configured to facilitatetermination of the remote session.

30. The system of clause 26, wherein the host trusted relationshipmodule is configured to signal the host remote session terminationmodule to terminate the remote session.

31. The system of clause 26, wherein the system is the host devicecomprising a processing system and a display.

32. The system of clause 26, wherein the host device is configured tocommunicate with the client device over a wireless connection.

33. The system of clause 27, wherein the host presence sensing module isconfigured to estimate received signal strength of a transmissionreceived from the client device.

34. The system of clause 27, wherein the host presence sensing module isconfigured to calculate a round trip delay time for communication withthe client device.

34A. The system of clause 26, wherein the system is the host device.

35. A method implemented at a host device side for communication and forfacilitating establishing a remote session between a host device and aremote server, the method comprising:

facilitating establishing a trusted relationship between a client deviceand the host device (e.g., 4300-A of FIG. 43);

obtaining remote session login information from the client device (e.g.,4302-A of FIG. 43); and

facilitating establishing a remote session, using the remote sessionlogin information, with the remote server (e.g., 4304-A of FIG. 43),

wherein the host device is configured to communicate with the clientdevice via a wireless connection.

36. The method of clause 35, wherein the facilitating establishing thetrusted relationship comprises:

sensing a presence of the client device near the host device;

pairing the client device with the host device; and

performing exchange of encryption information with the client device.

37. The method of clause 35, wherein the facilitating establishing thetrusted relationship comprises monitoring for continued existence of thetrusted relationship with the client device.

38. The method of clause 35, further comprising:

terminating the remote session.

39. The method of clause 35, wherein the terminating the remote sessioncomprises generating a signal to terminate the remote session.

40. The method of clause 36, wherein the sensing the presence comprisesestimating a received signal strength of a transmission received fromthe client device.

41. The method of clause 36, wherein the sensing the presence comprisescalculating a round trip delay time for communication with the clientdevice.

42. A machine-readable medium encoded with instructions for execution ata host device side for communication and for facilitating establishing aremote session between a host device and a remote server, theinstructions comprising code for:

facilitating establishing a trusted relationship between a client deviceand the host device;

obtaining remote session login information from the client device; and

facilitating establishing a remote session, using the remote sessionlogin information, with the remote server,

wherein the host device is configured to communicate with the clientdevice via a wireless connection.

43. The machine-readable medium of clause 42, wherein the code forfacilitating establishing the trusted relationship comprises code for:

sensing a presence of the client device near the host device;

pairing the client device with the host device; and

performing exchange of encryption information with the client device.

44. The machine-readable medium of clause 42, wherein the code forfacilitating establishing the trusted relationship comprises code formonitoring for continued existence of the trusted relationship with theclient device.

45. The machine-readable medium of clause 42, wherein the instructionsfurther comprise code for:

terminating the remote session.

46. The machine-readable medium of clause 42, wherein the terminatingthe remote session comprises generating a signal to terminate the remotesession.

47. The machine-readable medium of clause 43, wherein the sensing thepresence comprises estimating a received signal strength of atransmission received from the client device.

48. The machine-readable medium of clause 43, wherein the sensing thepresence comprises calculating a round trip delay time for communicationwith the client device.

49. An apparatus for communication and for facilitating establishing aremote session between a host device and a remote server, the apparatuscomprising:

means for facilitating establishing a trusted relationship between aclient device and the host device (e.g., 4300-B of FIG. 45);

means for obtaining remote session login information from the clientdevice (e.g., 4300-B of FIG. 45); and

means for facilitating establishing a remote session, using the remotesession login information, with the remote server (e.g., 4300-B of FIG.45);

wherein the host device is configured to communicate with the clientdevice via a wireless connection.

50. The apparatus of clause 49, wherein the means for facilitatingestablishing the trusted relationship comprises:

means for sensing a presence of the client device near the host device;

means for pairing the client device with the host device; and

means for performing exchange of encryption information with the clientdevice.

51. The apparatus of clause 49, wherein the means for facilitatingestablishing the trusted relationship comprises means for monitoring forcontinued existence of the trusted relationship with the client device.

52. The apparatus of clause 49, further comprising:

means for terminating the remote session.

53. The apparatus of clause 49, wherein the means for terminating theremote session comprises means for generating a signal to terminate theremote session.

54. The apparatus of clause 50, wherein the means for sensing thepresence comprises means for estimating a received signal strength of atransmission received from the client device.

55. The apparatus of clause 50, wherein the means for sensing thepresence comprises means for calculating a round trip delay time forcommunication with the client device.

55A. The apparatus of clause 49, wherein the apparatus is the hostdevice.

The subject technology is illustrated, for example, according to variousaspects described below. Numbered clauses are provided below forconvenience. These are provided as examples, and do not limit thesubject technology.

1. A system at a client device side for communication and forfacilitating establishing a remote session between a host device and aremote server, the system comprising:

a client session launch module configured to facilitate establishing afirst remote session between a client device and the remote server;

a client trusted relationship module configured to facilitateestablishing a trusted relationship between the client device and thehost device;

a client remote session login information module configured to provideremote session login information from the client device to the hostdevice to enable the host device to establish a second remote sessionwith the remote server; and

a client remote session termination module configured to facilitatetermination of the first remote session at the client device after thelogin information is provided to the host device.

2. The system of clause 1, wherein the client remote session terminationmodule is configured to facilitate the termination in response toreceiving an indication regarding an establishment of the second remotesession from the host device.

3. The system of clause 1, wherein the client trusted relationshipmodule is configured to respond to a query message from the host device.

4. The system of clause 1, wherein the client device is configured tocommunicate with the host device over a wireless connection.

5. The system of clause 1, wherein the client trusted relationshipmodule comprises:

a client presence sensing module configured to sense a presence of theclient device near the host device;

a client pairing module configured to facilitate performing messageexchanges to pair the client device with the host device; and

a client link security module configured to perform exchange ofencryption information with the host device.

6. The system of clause 5, wherein the client presence sensing module isconfigured to estimate a received signal strength of a transmissionreceived from the host device.

7. The system of clause 5, wherein the client presence sensing module isconfigured to calculate a round trip delay time for communication withthe host device.

8. The system of clause 1, wherein the system is the client devicecomprising a processing system and a display.

9. A method at a client device side for communication and forfacilitating establishing a remote session between a host device and aremote server, the method comprising:

facilitating establishing a first remote session between a client deviceand the remote server (e.g., 4600-A of FIG. 46);

facilitating establishing a trusted relationship between the clientdevice and the host device (e.g., 4602-A of FIG. 46);

providing remote session login information from the client device to thehost device to enable the host device to establish a second remotesession with the remote server (e.g., 4604-A of FIG. 46); and

facilitating termination of the first remote session at the clientdevice after the login information is provided to the host device (e.g.,4606-A of FIG. 46).

10. The method of clause 9, wherein the facilitating termination of thefirst remote session comprises facilitating the termination in responseto receiving an indication regarding an establishment of the secondremote session from the host device.

11. The method of clause 9, wherein the facilitating establishing thetrusted relationship comprises responding to a query message from thehost device.

12. The method of clause 9, wherein the client device communicates withthe host device over a wireless connection.

13. The method of clause 9, wherein the facilitating establishing thetrusted relationship comprises:

sensing a presence of the client device near the host device;

performing message exchanges to pair the client device with the hostdevice; and

performing exchange of encryption information with the host device.

14. The method of clause 13, wherein the sensing the presence comprisesestimating a received signal strength of a transmission received fromthe host device.

15. The method of clause 13, wherein the sensing the presence comprisescalculating a round trip delay time for communication with the hostdevice.

16. A machine-readable medium encoded with instructions for execution ata client device side for communication and for facilitating establishinga remote session between a host device and a remote server, theinstructions comprising code for:

facilitating establishing a first remote session between a client deviceand the remote server;

facilitating establishing a trusted relationship between the clientdevice and the host device;

providing remote session login information from the client device to thehost device to enable the host device to establish a second remotesession with the remote server; and

facilitating termination of the first remote session at the clientdevice after the login information is provided to the host device.

17. The machine-readable medium of clause 16, wherein the code forfacilitating termination of the first remote session comprises code forfacilitating the termination in response to receiving an indicationregarding an establishment of the second remote session from the hostdevice.

18. The machine-readable medium of clause 16, wherein the code forfacilitating establishing the trusted relationship comprises code forresponding to a query message from the host device.

19. The machine-readable medium of clause 16, wherein the client deviceis configured to communicate with the host device over a wirelessconnection.

20. The machine-readable medium of clause 16, wherein the code forfacilitating establishing the trusted relationship comprises code for:

sensing a presence of the client device near the host device;

performing message exchanges to pair the client device with the hostdevice; and

performing exchange of encryption information with the host device.

21. The machine-readable medium of clause 20, wherein the code forsensing the presence comprises code for estimating a received signalstrength of a transmission received from the host device.

22. The machine-readable medium of clause 20, wherein the code forsensing the presence comprises code for calculating a round trip delaytime for communication with the host device.

23. An apparatus for communication and for facilitating establishing aremote session between a host device and a remote server, the apparatuscomprising:

means for facilitating establishing a first remote session between aclient device and the remote server (e.g., 4600-B of FIG. 47);

means for facilitating establishing a trusted relationship between theclient device and the host device (e.g., 4602-B of FIG. 47);

means for providing remote session login information from the clientdevice to the host device to enable the host device to establish asecond remote session with the remote server (e.g., 4604-B of FIG. 47);and

means for facilitating termination of the first remote session at theclient device after the login information is provided to the host device(e.g., 4606-B of FIG. 47).

24. The apparatus of clause 23, wherein the means for facilitatingtermination of the first remote session comprises means for facilitatingthe termination in response to receiving an indication regarding anestablishment of the second remote session from the host device.

25. The apparatus of clause 23, wherein the means for facilitatingestablishing a trusted relationship comprises means for responding to aquery message from the host device.

26. The apparatus of clause 23, wherein the client device is configuredto communicate with the host device over a wireless connection.

27. The apparatus of clause 23, wherein the means for facilitatingestablishing the trusted relationship comprises:

means for sensing a presence of the client device near the host device;

means for performing message exchanges to pair the client device withthe host device; and

means for performing exchange of encryption information with the hostdevice.

28. The apparatus of clause 27, wherein the means for sensing thepresence comprises means for estimating a received signal strength of atransmission received from the host device.

29. The apparatus of clause 27, wherein the means for sensing thepresence comprises means for calculating a round trip delay time forcommunication with the host device.

29A. The apparatus of clause 23, wherein the apparatus is the clientdevice.

The subject technology is illustrated, for example, according to variousaspects described below. Numbered clauses are provided below forconvenience. These are provided as examples, and do not limit thesubject technology.

1. A system at a client device side for communication and for forwardingdisplay data related to a remote session between a client device and aremote server to a host device, the system comprising:

a client remote session module configured to facilitate establishing theremote session with the remote server;

a client trusted relationship module configured to facilitateestablishing a trusted relationship between the client device and thehost device; and

a client forwarding module configured to filter out data related tolocal graphical user interface (GUI) and selectively forward from theclient device to the host device display data related to the remotesession established between the client device and the remote server.

2. The system of clause 1, wherein the client device is configured tocommunicate with the host device via a first network connection and withthe remote server via a second network connection.

3. The system of clause 1, wherein the client trusted relationshipmodule comprises:

a client presence sensing module configured to sense a presence of theclient device near the host device;

a client pairing module configured to facilitate performing messageexchanges to pair the client device with the host device; and

a client link security module configured to perform exchange ofencryption information with the host device.

4. The system of clause 1, wherein the client trusted relationshipmodule is configured to respond to a query message from the host device.

5. The system of clause 1, wherein the client device is configured tocommunicate with the host device over a wireless connection.

6. The system of clause 3, wherein the client presence sensing module isconfigured to estimate a received signal strength of a transmissionreceived from the host device.

7. The system of clause 3, wherein the client presence sensing module isconfigured to calculate a round trip delay time for communication withthe host device.

8. The system of clause 1, wherein the system is the client devicecomprising a processing system and a display.

9. A method at a client device side for communication and for forwardingdisplay data related to a remote session between a client device and aremote server to a host device, the method comprising:

facilitating establishing the remote session with the remote server(e.g., 4800-A of FIG. 48);

facilitating establishing a trusted relationship between the clientdevice and the host device (e.g., 4802-A of FIG. 48); and

filtering out data related to local graphical user interface (GUI)(e.g., 4804-A of FIG. 48); and

selectively forwarding, from the client device to the host device,display data related to the remote session established between theclient device and the remote server (e.g., 4806-A of FIG. 48).

10. The method of clause 9, wherein the client device communicates withthe host device via a first network connection and with the remoteserver via a second network connection.

11. The method of clause 9, wherein the facilitating establishing thetrusted relationship comprises:

sensing a presence of the client device near the host device;

performing message exchanges to pair the client device with the hostdevice; and

performing exchange of encryption information with the host device.

12. The method of clause 9, wherein the facilitating establishing thetrusted relationship comprises responding to a query message from thehost device.

13. The method of clause 9, wherein the client device communicates withthe host device over a wireless connection.

14. The method of clause 11, wherein the sensing the presence comprisesestimating a received signal strength of a transmission received fromthe host device.

15. The method of clause 11, wherein the sensing the presence comprisescalculating a round trip delay time for communication with the hostdevice.

16. A machine-readable medium encoded with instructions for execution ata client device side for communication and for forwarding display datarelated to a remote session between a client device and a remote serverto a host device, the instructions comprising code for:

facilitating establishing the remote session with the remote server;

facilitating establishing a trusted relationship between the clientdevice and the host device; and

filtering out data related to local graphical user interface (GUI); and

selectively forwarding, from the client device to the host device,display data related to the remote session established between theclient device and the remote server.

17. The machine-readable medium of clause 16, wherein the client deviceis configured to communicate with the host device via a first networkconnection and with the remote server via a second network connection.

18. The machine-readable medium of clause 16, wherein the code forfacilitating establishing the trusted relationship comprises code for:

sensing a presence of the client device near the host device;

performing message exchanges to pair the client device with the hostdevice; and

performing exchange of encryption information with the host device.

19. The machine-readable medium of clause 16, wherein the code forfacilitating establishing the trusted relationship comprises code forresponding to a query message from the host device.

20. The machine-readable medium of clause 16, wherein the client deviceis configured to communicate with the host device over a wirelessconnection.

21. The machine-readable medium of clause 18, wherein the code forsensing the presence comprises code for estimating a received signalstrength of a transmission received from the host device.

22. The machine-readable medium of clause 18, wherein the code forsensing the presence comprises code for calculating a round trip delaytime for communication with the host device.

23. An apparatus for communication and for forwarding display datarelated to a remote session between a client device and a remote serverto a host device, the apparatus comprising:

means for facilitating establishing the remote session with the remoteserver (e.g., 4800-B of FIG. 50);

means for facilitating establishing a trusted relationship between theclient device and the host device (e.g., 4802-B of FIG. 50); and

means for filtering out data related to local graphical user interface(GUI) (e.g., 4804-B of FIG. 50); and

means for selectively forwarding, from the client device to the hostdevice, display data related to the remote session established betweenthe client device and the remote server (e.g., 4806-B of FIG. 50).

24. The apparatus of clause 23, wherein the client device is configuredto communicate with the host device via a first network connection andwith the remote server via a second network connection.

25. The apparatus of clause 23, wherein the means for facilitatingestablishing the trusted relationship comprises:

means for sensing a presence of the client device near the host device;

means for performing message exchanges to pair the client device withthe host device; and

means for performing exchange of encryption information with the hostdevice.

26. The apparatus of clause 23, wherein the means for facilitatingestablishing the trusted relationship comprises means for responding toa query message from the host device.

27. The apparatus of clause 23, wherein the client device is configuredto communicate with the host device over a wireless connection.

28. The apparatus of clause 25, wherein the means for sensing thepresence comprises means for estimating a received signal strength of atransmission received from the host device.

29. The apparatus of clause 25, wherein the means for sensing thepresence comprises means for calculating a round trip delay time forcommunication with the host device.

29A. The apparatus of clause 23, wherein the apparatus is the clientdevice.

30. A system at a host device side for communication and for displayinggraphical user interface (GUI) for a remote session between a clientdevice and a remote server, comprising:

a host trusted relationship module configured to facilitate establishinga trusted relationship between the client device and the host device;

a host display data module configured to receive, from the clientdevice, display data for the remote session established between theclient device and the remote server, the display data based on a remotesession protocol format; and

a host remote session view module configured to facilitate displaying aremote application view based on the received display data for theremote session,

wherein the host device is configured to communicate with the clientdevice via a wireless connection.

31. The system of clause 30, wherein the remote application viewcomprises graphical user interface control objects.

32. The system of clause 30, wherein the host trusted relationshipmodule comprises:

a host presence sensing module configured to sense a presence of theclient device near the host device;

a host pairing module configured to pair the client device with the hostdevice; and

a host link security module configured to perform exchange of encryptioninformation with the host device.

33. The system of clause 30, wherein the host device is configured tocommunicate with the client device over a wireless connection.

34. The system of clause 32, wherein the host presence sensing module isconfigured to estimate received signal strength of a transmissionreceived from the client device.

35. The system of clause 32, wherein the host presence sensing module isconfigured to calculate a round trip delay time for communication withthe client device.

36. The system of clause 30, wherein the system is the host devicecomprising a processing system and a display.

37. A method at a host device side for communication and for displayinggraphical user interface (GUI) for a remote session between a clientdevice and a remote server, the method comprising:

facilitating establishing a trusted relationship between the clientdevice and the host device (e.g., 4900-A of FIG. 49);

receiving, from the client device, display data for the remote sessionestablished between the client device and the remote server, the displaydata based on a remote session protocol format (e.g., 4902-A of FIG.49); and

facilitating displaying a remote application view based on the receiveddisplay data for the remote session (e.g., 4904-A of FIG. 49);

wherein the host device is configured to communicate with the clientdevice via a wireless connection.

38. The method of clause 37, wherein the remote application viewcomprises graphical user interface control objects.

39. The method of clause 37, wherein the facilitating establishing thetrusted relationship comprises:

sensing a presence of the client device near the host device;

pairing the client device with the host device; and

performing exchange of encryption information with the host device.

40. The method of clause 37, wherein the host device is configured tocommunicate with the client device over a wireless connection.

41. The method of clause 39, wherein the sensing the presence comprisesestimating received signal strength of a transmission received from theclient device.

42. The method of clause 39, wherein the sensing the presence comprisescalculating a round trip delay time for communication with the clientdevice.

43. A machine-readable medium encoded with instructions for execution ata host device side for communication and for displaying graphical userinterface (GUI) for a remote session between a client device and aremote server, the instructions comprising code for:

facilitating establishing a trusted relationship between the clientdevice and the host device;

receiving, from the client device, display data for the remote sessionestablished between the client device and the remote server, the displaydata based on a remote session protocol format; and

facilitating displaying a remote application view based on the receiveddisplay data for the remote session,

wherein the host device is configured to communicate with the clientdevice via a wireless connection.

44. The machine-readable medium of clause 43, wherein the remoteapplication view comprises graphical user interface control objects.

45. The machine-readable medium of clause 43, wherein the code forfacilitating establishing the trusted relationship comprises code for:

sensing a presence of the client device near the host device;

pairing the client device with the host device; and

performing exchange of encryption information with the host device.

46. The machine-readable medium of clause 43, wherein the host device isconfigured to communicate with the client device over a wirelessconnection.

47. The machine-readable medium of clause 45, wherein the sensing thepresence comprises estimating received signal strength of a transmissionreceived from the client device.

48. The machine-readable medium of clause 45, wherein the sensing thepresence comprises calculating a round trip delay time for communicationwith the client device.

49. An apparatus for communication and for displaying graphical userinterface (GUI) for a remote session between a client device and aremote server, the apparatus comprising:

means for facilitating establishing a trusted relationship between theclient device and the host device (e.g., 4900-B of FIG. 51);

means for receiving, from the client device, display data for the remotesession established between the client device and the remote server, thedisplay data based on a remote session protocol format (e.g., 4902-B ofFIG. 51); and

means for facilitating displaying a remote application view based on thereceived display data for the remote session (e.g., 4904-B of FIG. 51);

wherein the host device is configured to communicate with the clientdevice via a wireless connection.

50. The apparatus of clause 49, wherein the remote application viewcomprises graphical user interface control objects.

51. The apparatus of clause 49, wherein the means for facilitatingestablishing the trusted relationship comprises:

means for sensing a presence of the client device near the host device;

means for pairing the client device with the host device; and

means for performing exchange of encryption information with the hostdevice.

52. The apparatus of clause 49, wherein the host device is configured tocommunicate with the client device over a wireless connection.

53. The apparatus of clause 51, wherein the means for sensing thepresence comprises means for estimating received signal strength of atransmission received from the client device.

54. The apparatus of clause 51, wherein the means for sensing thepresence comprises means for calculating a round trip delay time forcommunication with the client device.

54A. The apparatus of clause 49, wherein the apparatus is the hostdevice.

The subject technology is illustrated, for example, according to variousaspects described below. Numbered clauses are provided below forconvenience. These are provided as examples, and do not limit thesubject technology.

1. A system at a client device side for communication and forfacilitating establishing a remote session between a client device and aremote server, the system comprising:

a client trusted relationship module configured to facilitateestablishing a trusted relationship between the client device and a hostdevice;

a client login information receiver module configured to receive logininformation from the host device for a first remote session establishedbetween the host device and the remote server; and

a client session launch module configured to facilitate continuing thefirst remote session previously established between the host device andthe remote server as a continued remote session between the clientdevice and the remote server.

2. The system of clause 1, wherein the client trusted relationshipmodule comprises:

a client presence sensing module configured to sense presence of theclient device near the host device;

a client pairing module configured to facilitate performing messageexchange to pair the client device with the host device; and

a client link security module configured to perform exchange ofencryption information with the host device.

3. The system of clause 1, wherein the client trusted relationshipmodule is configured to respond to a query message from the host device.

4. The system of clause 1, wherein the client device is configured tocommunicate with the host device over a wireless connection.

5. The system of clause 2, wherein the client presence sensing module isconfigured to estimate a received signal strength of a transmissionreceived from the host device.

6. The system of clause 2, wherein the client presence sensing module isconfigured to calculate a round trip delay time for communication withthe host device.

7. The system of clause 1, wherein the system is the client devicecomprising a processing system and a display.

8. A method at a client device side for communication and forfacilitating establishing a remote session between a client device and aremote server, the method comprising:

facilitating establishing a trusted relationship between the clientdevice and a host device (e.g., 5200-A of FIG. 52);

receiving login information from the host device for a first remotesession established between the host device and the remote server (e.g.,5202-A of FIG. 52); and

continuing the first remote session previously established between thehost device and the remote server as a continued remote session betweenthe client device and the remote server (e.g., 5204-A of FIG. 52).

9. The method of clause 8, wherein the facilitating establishing thetrusted relationship comprises:

sensing presence of the client device near the host device;

facilitating performing message exchange to pair the client device withthe host device; and

performing exchange of encryption information with the host device.

10. The method of clause 8, wherein the facilitating establishing thetrusted relationship comprises responding to a query message from thehost device.

11. The method of clause 8, wherein the client device communicates withthe host device over a wireless connection.

12. The method of clause 9, wherein the sensing presence comprisesestimating a received signal strength of a transmission received fromthe host device.

13. The method of clause 9, wherein the sensing presence comprisescalculating a round trip delay time for communication with the hostdevice.

14. A machine-readable medium encoded with instructions for execution ata client device side for communication and for facilitating establishinga remote session between a client device and a remote server, theinstructions comprising code for:

facilitating establishing a trusted relationship between the clientdevice and a host device;

receiving login information from the host device for a first remotesession established between the host device and the remote server; and

continuing the first remote session previously established between thehost device and the remote server as a continued remote session betweenthe client device and the remote server.

15. The machine-readable medium of clause 14, wherein the code forfacilitating establishing the trusted relationship comprises code for:

sensing presence of the client device near the host device;

facilitating performing message exchange to pair the client device withthe host device; and

performing exchange of encryption information with the host device.

16. The machine-readable medium of clause 14, wherein the code forfacilitating establishing the trusted relationship comprises code forresponding to a query message from the host device.

17. The machine-readable medium of clause 14, wherein the client deviceis configured to communicate with the host device over a wirelessconnection.

18. The machine-readable medium of clause 15, wherein the code forsensing presence comprises code for estimating a received signalstrength of a transmission received from the host device.

19. The machine-readable medium of clause 15, wherein the code forsensing presence comprises calculating a round trip delay time forcommunication with the host device.

20. An apparatus at a client device side for communication and forfacilitating establishing a remote session between a client device and aremote server, comprising:

means for facilitating establishing a trusted relationship between theclient device and a host device (e.g., 5200-B of FIG. 54);

means for receiving login information from the host device for a firstremote session established between the host device and the remote server(e.g., 5202-B of FIG. 54); and

means for continuing the first remote session previously establishedbetween the host device and the remote server as a continued remotesession between the client device and the remote server (e.g., 5204-B ofFIG. 54).

21. The apparatus of clause 20, wherein the means for facilitatingestablishing the trusted relationship comprises:

means for sensing presence of the client device near the host device;

means for facilitating performing message exchange to pair the clientdevice with the host device; and

means for performing exchange of encryption information with the hostdevice.

22. The apparatus of clause 20, wherein the means for facilitatingestablishing the trusted relationship comprises means for responding toa query message from the host device.

23. The apparatus of clause 20, wherein the client device is configuredto communicate with the host device over a wireless connection.

24. The apparatus of clause 21, wherein the means for sensing presencecomprises means for estimating a received signal strength of atransmission received from the host device.

25. The apparatus of clause 21, wherein the means for sensing presencecomprises means for calculating a round trip delay time forcommunication with the host device.

25A. The apparatus of clause 20, wherein the apparatus is the clientdevice.

26. A system at a host device side for communication and fortransferring a remote session with a remote server to a client device,comprising:

a host remote session establishment module configured to facilitateestablishing the remote session between the host device and the remoteserver;

a host trusted relationship module configured to facilitate establishinga trusted relationship between the client device and the host device;

a host login information module configured to provide login informationfor the remote session to the client device; and

a host remote session termination module configured to terminate theremote session running at the host device after providing the logininformation.

27. The system of clause 26, wherein the host remote session terminationmodule is configured to facilitate the termination in response toreceiving an indication regarding an establishment of a second remotesession from the client device.

28. The system of clause 26, wherein the host trusted relationshipmodule comprises:

a host presence sensing module configured to sense a presence of theclient device near the host device;

a host pairing module configured to pair the client device with the hostdevice; and

a host link security module configured to perform exchange of encryptioninformation with the host device.

29. The system of clause 26, wherein the host device is configured tocommunicate with the client device over a wireless connection.

30. The system of clause 28, wherein the host presence sensing module isconfigured to estimate a received signal strength of a transmissionreceived from the client device.

31. The system of clause 28, wherein the host presence sensing module isconfigured to calculate a round trip delay time for communication withthe client device.

32. The system of clause 26, wherein the system is the host devicecomprising a processing system and a display.

33. A method at a host device side for transferring a remote sessionwith a remote server to a client device, comprising:

facilitating establishing the remote session between the host device andthe remote server (e.g., 5300-A of FIG. 53);

facilitating establishing a trusted relationship between the clientdevice and the host device (e.g., 5302-A of FIG. 53);

providing login information for the remote session to the client device(e.g., 5304-A of FIG. 53); and

terminating the remote session running at the host device afterproviding the login information (e.g., 5306-A of FIG. 53).

34. The method of clause 33, wherein the terminating the remote sessioncomprises facilitating termination in response to receiving anindication regarding an establishment of a second remote session fromthe client device.

35. The method of clause 33, wherein the facilitating establishing thetrusted relationship comprises:

sensing a presence of the client device near the host device;

pairing the client device with the host device; and

performing exchange of encryption information with the host device.

36. The method of clause 33, wherein the host device is configured tocommunicate with the client device over a wireless connection.

37. The method of clause 35, wherein the sensing the presence comprisesestimating a received signal strength of a transmission received fromthe client device.

38. The method of clause 35, wherein the sensing the presence comprisescalculating a round trip delay time for communication with the clientdevice.

39. A machine-readable medium encoded with instructions for execution ata host device side for transferring a remote session with a remoteserver to a client device, the instructions comprising code for:

facilitating establishing the remote session between the host device andthe remote server;

facilitating establishing a trusted relationship between the clientdevice and the host device;

providing login information for the remote session to the client device;and

terminating the remote session running at the host device afterproviding the login information.

40. The machine-readable medium of clause 39, wherein the code forterminating the remote session comprises code for facilitatingtermination in response to receiving an indication regarding anestablishment of a second remote session from the client device.

41. The machine-readable medium of clause 39, wherein the code forfacilitating establishing the trusted relationship comprises code for:

sensing a presence of the client device near the host device;

pairing the client device with the host device; and

performing exchange of encryption information with the host device.

42. The machine-readable medium of clause 39, wherein the host device isconfigured to communicate with the client device over a wirelessconnection.

43. The machine-readable medium of clause 41, wherein the code forsensing the presence comprises code for estimating a received signalstrength of a transmission received from the client device.

44. The machine-readable medium of clause 41, wherein the code forsensing the presence comprises code for calculating a round trip delaytime for communication with the client device.

45. An apparatus for transferring a remote session with a remote serverto a client device, comprising:

means for facilitating establishing the remote session between the hostdevice and the remote server (e.g., 5300-B of FIG. 55);

means for facilitating establishing a trusted relationship between theclient device and the host device (e.g., 5302-B of FIG. 55);

means for providing login information for the remote session to theclient device (e.g., 5304-B of FIG. 55); and

means for terminating the remote session running at the host deviceafter providing the login information (e.g., 5306-B of FIG. 55).

46. The apparatus of clause 45, wherein the means for terminating theremote session comprises means for facilitating termination in responseto receiving an indication regarding an establishment of a second remotesession from the client device.

47. The apparatus of clause 45, wherein the means for facilitatingestablishing the trusted relationship comprises:

means for sensing a presence of the client device near the host device;

means for pairing the client device with the host device; and

means for performing exchange of encryption information with the hostdevice.

48. The apparatus of clause 45, wherein the host device is configured tocommunicate with the client device over a wireless connection.

49. The apparatus of clause 47, wherein the means for sensing thepresence comprises means for estimating a received signal strength of atransmission received from the client device.

50. The apparatus of clause 47, wherein the means for sensing thepresence comprises means for calculating a round trip delay time forcommunication with the client device.

50A. The apparatus of clause 45, wherein the apparatus is the hostdevice.

Those of skill in the art would appreciate that the various illustrativeblocks, modules, elements, components, methods, and algorithms describedherein may be implemented as electronic hardware, computer software, orcombinations of both.

For example, a module (e.g., a client remote session login informationmodule 2604, a client session launch module 2606, a host trustedrelationship module 2702, or any other modules) may be implemented aselectronic hardware, computer software, or combinations of both. In oneaspect, a module(s) may be an apparatus since a module(s) may includeinstructions encoded or stored on a machine-readable medium, on anotherdevice, or on a portion thereof. In one aspect, a module(s) may besoftware (e.g., an application, a subroutine) stored in amachine-readable medium and executable by a processing system or aprocessor. In another aspect, a module(s) may be hardware (e.g.,machine-readable medium encoded with instructions, a pre-programmedgeneral-purpose computer, or a special purpose electronic or opticaldevice).

Various modules may reside in one machine or in multiple machines. Inone example, modules for the server side (e.g., a host trustedrelationship module 2702) may be located in one server or spread overmultiple servers. In another example, modules for the client side (e.g.,a client remote session login information module, a client sessionlaunch module 2606, etc.) may be located in one client device or spreadover multiple client devices.

In one aspect of the disclosure, when actions or functions are describedas being performed by a module or a component (e.g., establishing,sending, receiving, providing, building, displaying, registering,encrypting, decrypting, authenticating, notifying, accepting, selecting,controlling, issuing, transmitting, reporting, pushing, or any otheraction or function), it is understood that such actions or functions areperformed by the module or the component directly or indirectly. As anexample, when a module is described as performing an action, it isunderstood that the module may perform the action directly or mayperform the action indirectly, for example, by facilitating such anaction. For instance, when a session is described as being establishedby a module, it is understood that the module may establish the sessionindirectly by facilitating an establishment of the session. As yetanother example, when a view of an application is described as beingdisplayed or rendered by a module, it is understood that the view may bedisplayed or rendered by the module either directly or indirectly.

To illustrate this interchangeability of hardware and software, variousillustrative blocks, modules, elements, components, methods, andalgorithms have been described above generally in terms of theirfunctionality. Whether such functionality is implemented as hardware orsoftware depends upon the particular application and design constraintsimposed on the overall system. Skilled artisans may implement thedescribed functionality in varying ways for each particular application.

Various components and blocks may be arranged differently (e.g.,arranged in a different order, or partitioned in a different way) allwithout departing from the scope of the subject technology. In oneaspect of the disclosure, the modules (or elements) recited in theaccompanying claims may be performed by one module or by a smallernumber of modules, and this arrangement is within the scope of theclaims. In another aspect, the modules (or elements) recited in theaccompanying claims may be performed by a larger number of modules, andthis arrangement is within the scope of the claims. In yet anotheraspect, a module (or an element) recited in the accompanying claims maybe performed by multiple modules, and this arrangement is within thescope of the claims. For example, a local view module and a remote viewmodule may be combined into one module. A client remote access module, alocal view module and a remote view module may be combined into onemodule. An agent module and a server remote access module may becombined into one module. In another example, these modules may bedivided into a larger number of modules.

It is understood that the specific order or hierarchy of steps in theprocesses disclosed is an illustration of exemplary approaches. Basedupon design preferences, it is understood that the specific order orhierarchy of steps in the processes may be rearranged. Some of the stepsmay be performed simultaneously. The accompanying method claims presentelements of the various steps in a sample order, and are not meant to belimited to the specific order or hierarchy presented.

The previous description is provided to enable any person skilled in theart to practice the various aspects described herein. The previousdescription provides various examples of the subject technology, and thesubject technology is not limited to these examples. Variousmodifications to these aspects will be readily apparent to those skilledin the art, and the generic principles defined herein may be applied toother aspects. Thus, the claims are not intended to be limited to theaspects shown herein, but is to be accorded the full scope consistentwith the language claims, wherein reference to an element in thesingular is not intended to mean “one and only one” unless specificallyso stated, but rather “one or more.” Unless specifically statedotherwise, the term “some” refers to one or more. Pronouns in themasculine (e.g., his) include the feminine and neuter gender (e.g., herand its) and vice versa. Headings and subheadings, if any, are used forconvenience only and do not limit the invention.

A phrase such as an “aspect” does not imply that such aspect isessential to the subject technology or that such aspect applies to allconfigurations of the subject technology. A disclosure relating to anaspect may apply to all configurations, or one or more configurations.An aspect may provide one or more examples of the disclosure. A phrasesuch as an aspect may refer to one or more aspects and vice versa. Aphrase such as an “embodiment” does not imply that such embodiment isessential to the subject technology or that such embodiment applies toall configurations of the subject technology. A disclosure relating toan embodiment may apply to all embodiments, or one or more embodiments.An embodiment may provide one or more examples of the disclosure. Aphrase such an embodiment may refer to one or more embodiments and viceversa. A phrase such as a “configuration” does not imply that suchconfiguration is essential to the subject technology or that suchconfiguration applies to all configurations of the subject technology. Adisclosure relating to a configuration may apply to all configurations,or one or more configurations. A configuration may provide one or moreexamples of the disclosure. A phrase such a configuration may refer toone or more configurations and vice versa.

The word “exemplary” is used herein to mean “serving as an example orillustration.” Any aspect or design described herein as “exemplary” isnot necessarily to be construed as preferred or advantageous over otheraspects or designs.

All structural and functional equivalents to the elements of the variousaspects described throughout this disclosure that are known or latercome to be known to those of ordinary skill in the art are expresslyincorporated herein by reference and are intended to be encompassed bythe claims. Moreover, nothing disclosed herein is intended to bededicated to the public regardless of whether such disclosure isexplicitly recited in the claims. No claim element is to be construedunder the provisions of 35 U.S.C. §112, sixth paragraph, unless theelement is expressly recited using the phrase “means for” or, in thecase of a method claim, the element is recited using the phrase “stepfor.” Furthermore, to the extent that the term “include,” “have,” or thelike is used in the description or the claims, such term is intended tobe inclusive in a manner similar to the term “comprise” as “comprise” isinterpreted when employed as a transitional word in a claim.

1. A system at a client device side for communication and for forwardingdisplay data related to a remote session between a client device and aremote server to a host device, the system comprising: a client remotesession module configured to facilitate establishing the remote sessionwith the remote server; a client trusted relationship module configuredto facilitate establishing a trusted relationship between the clientdevice and the host device; and a client forwarding module configured tofilter out data related to local graphical user interface (GUI) andselectively forward from the client device to the host device displaydata related to the remote session established between the client deviceand the remote server.
 2. A method at a client device side forcommunication and for forwarding display data related to a remotesession between a client device and a remote server to a host device,the method comprising: facilitating establishing the remote session withthe remote server; facilitating establishing a trusted relationshipbetween the client device and the host device; and filtering out datarelated to local graphical user interface (GUI); and selectivelyforwarding, from the client device to the host device, display datarelated to the remote session established between the client device andthe remote server.
 3. A machine-readable medium encoded withinstructions for execution at a client device side for communication andfor forwarding display data related to a remote session between a clientdevice and a remote server to a host device, the instructions comprisingcode for: facilitating establishing the remote session with the remoteserver; facilitating establishing a trusted relationship between theclient device and the host device; and filtering out data related tolocal graphical user interface (GUI); and selectively forwarding, fromthe client device to the host device, display data related to the remotesession established between the client device and the remote server. 4.The machine-readable medium of claim 3, wherein the client device isconfigured to communicate with the host device via a first networkconnection and with the remote server via a second network connection.5. The machine-readable medium of claim 3, wherein the code forfacilitating establishing the trusted relationship comprises code for:sensing a presence of the client device near the host device; performingmessage exchanges to pair the client device with the host device; andperforming exchange of encryption information with the host device. 6.The machine-readable medium of claim 3, wherein the code forfacilitating establishing the trusted relationship comprises code forresponding to a query message from the host device.
 7. Themachine-readable medium of claim 3, wherein the client device isconfigured to communicate with the host device over a wirelessconnection.
 8. The machine-readable medium of claim 5, wherein the codefor sensing the presence comprises code for estimating a received signalstrength of a transmission received from the host device.
 9. Themachine-readable medium of claim 5, wherein the code for sensing thepresence comprises code for calculating a round trip delay time forcommunication with the host device.
 10. An apparatus for communicationand for forwarding display data related to a remote session between aclient device and a remote server to a host device, the apparatuscomprising: means for facilitating establishing the remote session withthe remote server; means for facilitating establishing a trustedrelationship between the client device and the host device; and meansfor filtering out data related to local graphical user interface (GUI);and means for selectively forwarding, from the client device to the hostdevice, display data related to the remote session established betweenthe client device and the remote server.
 11. A system at a host deviceside for communication and for displaying graphical user interface (GUI)for a remote session between a client device and a remote server,comprising: a host trusted relationship module configured to facilitateestablishing a trusted relationship between the client device and thehost device; a host display data module configured to receive, from theclient device, display data for the remote session established betweenthe client device and the remote server, the display data based on aremote session protocol format; and a host remote session view moduleconfigured to facilitate displaying a remote application view based onthe received display data for the remote session, wherein the hostdevice is configured to communicate with the client device via awireless connection.
 12. A method at a host device side forcommunication and for displaying graphical user interface (GUI) for aremote session between a client device and a remote server, the methodcomprising: facilitating establishing a trusted relationship between theclient device and the host device; receiving, from the client device,display data for the remote session established between the clientdevice and the remote server, the display data based on a remote sessionprotocol format; and facilitating displaying a remote application viewbased on the received display data for the remote session; wherein thehost device is configured to communicate with the client device via awireless connection.
 13. A machine-readable medium encoded withinstructions for execution at a host device side for communication andfor displaying graphical user interface (GUI) for a remote sessionbetween a client device and a remote server, the instructions comprisingcode for: facilitating establishing a trusted relationship between theclient device and the host device; receiving, from the client device,display data for the remote session established between the clientdevice and the remote server, the display data based on a remote sessionprotocol format; and facilitating displaying a remote application viewbased on the received display data for the remote session, wherein thehost device is configured to communicate with the client device via awireless connection.
 14. The machine-readable medium of claim 13,wherein the remote application view comprises graphical user interfacecontrol objects.
 15. The machine-readable medium of claim 13, whereinthe code for facilitating establishing the trusted relationshipcomprises code for: sensing a presence of the client device near thehost device; pairing the client device with the host device; andperforming exchange of encryption information with the host device. 16.The machine-readable medium of claim 13, wherein the host device isconfigured to communicate with the client device over a wirelessconnection.
 17. The machine-readable medium of claim 15, wherein thesensing the presence comprises estimating received signal strength of atransmission received from the client device.
 18. The machine-readablemedium of claim 15, wherein the sensing the presence comprisescalculating a round trip delay time for communication with the clientdevice.
 19. An apparatus for communication and for displaying graphicaluser interface (GUI) for a remote session between a client device and aremote server, the apparatus comprising: means for facilitatingestablishing a trusted relationship between the client device and thehost device; means for receiving, from the client device, display datafor the remote session established between the client device and theremote server, the display data based on a remote session protocolformat; and means for facilitating displaying a remote application viewbased on the received display data for the remote session; wherein thehost device is configured to communicate with the client device via awireless connection.